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Executive Summary 


In 2007, the Texas Legislature (80th Texas Legislature, Regular Session, 2007) authorized the creation of 
the Texas Rural Technology (R-Tech) Pilot program, which provides $8 million in funding to support 
rural districts in implementing technology-based supplemental education programs. In order to be eligible 
for funding, districts must have served fewer than 5,000 students and must not have been located in a 
metropolitan region of the state in 2007. Districts with limited course offerings and low accountability 
ratings received priority in grant awards. R-Tech funding is intended to support supplemental educational 
programs, including online courses, offered outside of students’ regularly scheduled classes (e.g., before 
or after school). Districts that receive funding are required to provide students in Grades 6 through 12 
with access to technology-based instructional resources for a minimum of 10 hours a week. 

R-Tech grants were awarded in two periods, or cycles. The Texas Education Agency (TEA) awarded 
approximately $6.3 million in funding to 64 districts 1 in Cycle 1 grant awards, and $1.5 million in funding 
to 19 districts in Cycle 2 grant awards. 2 Cycle 1 grant awards must be used during the May 1, 2008, 
through May 31, 2010, project period, and Cycle 2 awards must be used during the January 1, 2009, 
through May 31, 2010, project period. Grantee districts receive $200 per student served by R-Tech in 
state funding for each year of the grant and are required to provide matching funds of $100 per student 
per grant year. 

In establishing R-Tech, the Legislature required that the program be evaluated to assess its effects on 
student and teacher outcomes, as well as the program’s cost effectiveness. In addressing these goals, the 
evaluation considers the following research questions: 

1. How is R-Tech implemented across grantee districts and schools? 

2. What is the level of student participation in R-Tech? 

3. What is the effect of R-Tech on teachers? 

4. What is the effect of R-Tech on student outcomes? 

5. How cost effective is R-Tech? 

The evaluation is made up of two interim reports (fall 2008 and winter 2010) and a final report (fall 
2010). The findings presented here are drawn from the evaluation’s second interim report (winter 2010). 
The report’s findings arc preliminary and consider outcomes from R-Tech’ s first implementation year for 
only those districts receiving Cycle 1 grant awards. The evaluation’s final report will provide more 
complete information about Cycle 1 districts’ experiences in implementing R-Tech for the full 2-year 
grant period. 

RESEARCH QUESTIONS: KEY FINDINGS 

The sections that follow present key findings relative to each of the evaluation’s research questions. 
Results arc preliminary and address outcomes for Cycle 1 districts for R-Tech’ s first implementation year. 

Research Question 1: How is R-Tech Implemented Across Grantee Districts and 
Schools? 

The following sections present information about the types of programs districts implemented using 
R-Tech funds, as well as principals’ and facilitators’ roles in implementing the program, the challenges to 
implementation and how challenges were overcome. 


'One Cycle 1 district opted not to participate in the grant, which reduced the total number of Cycle 1 grantees to 63. 
2 Three Cycle 2 districts also received Cycle 1 awards. 
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Supplemental vs. non-supplemental programs. Although R-Tech was intended to support districts’ 
efforts in implementing supplemental educational programs offered outside the regularly scheduled 
school day, a substantial proportion of Cycle 1 districts (40%) implemented R-Tech as paid of classroom 
instruction (i.e., non-supplemental programs). Many districts used R-Tech funding to update their 
computer labs, and teachers scheduled class time in the lab for students to access resources. Two districts 
implemented R-Tech as a technology immersion program and used funding to support the purchase of 
laptop computers for all teachers and students in Grades 6 through 12. Students and teachers use laptops 
throughout the school day and may take laptops home. 

While some districts planned non-supplemental programs (e.g., technology immersion programs), other 
districts encountered challenges in implementing supplemental programs that caused them to revise their 
plans. District representatives explained that many students resisted participating in programs offered 
before or after school. Further, some students were not able to participate in R-Tech services because of 
conflicts with extra-curricular activities and bus schedules that limited their ability to arrive early or stay 
after school. 

Self-paced instructional programs. Most districts (87%) 3 implemented R-Tech as a self-paced program 
focused on tutoring, remediation, or credit recovery. Self-paced programs provide access to online lessons 
that students work through at their own pace. Many self-paced programs include diagnostic assessments 
of students’ individual learning needs and tailor instruction based on assessment outcomes. Some 
programs enable students to complete entire courses online, allowing students to make up credit for 
incomplete or failed courses. Sixty percent of districts offering self-paced instructional programs 
implemented supplemental programs in which students accessed resources outside of regularly scheduled 
classes. 

Dual credit and distance learning. About 30% of Cycle 1 districts offered dual credit coursework using 
R-Tech funding. Dual credit courses enable students in Grades 1 1 and 12 to take courses that fulfill high 
school graduation requirements and earn college credit. Such courses are generally taught by college or 
university faculty and students participate online or through the use of video conferencing equipment. 
R-Tech districts implementing dual credit courses partnered with community colleges and universities to 
provide instruction, and some programs were facilitated by regional Education Service Centers (ESCs). 
Sixty percent of districts that offered dual credit programs offered supplemental programs in which 
students participated in dual credit courses in addition to their regularly scheduled classes. 

Other programs. Six Cycle 1 districts offered different types of programs. Two districts used R-Tech 
funding to purchase iPods, which were loaded with instructional content for students to use at home 
(supplemental programs). Two districts offered technology immersion programs in which all students 
received laptops to use as part of regular instruction (non-supplemental programs). Two other districts 
planned to offer R-Tech as a program that included one-to-one tutoring with online instructional support; 
however, neither district implemented its program for students during R-Tech’ s first year. It is not known 
whether R-Tech services will be implemented as supplemental or non-supplemental programs in these 
districts. 

Implementation roles. R-Tech facilitators had the largest role in implementing district programs. 
Principals primarily provided support for communicating program goals and planning for the grant, but 
had lesser roles in the day-to-day management of the R-Tech activities. In most districts, teachers had 
little or no role in planning and implementing R-Tech activities during the grant’s first year. 


3 The percentage of districts included in each program type will not total to 100 because districts were able to 
implement more than one type of program. Districts were able to implement separate programs in their middle and 
high schools. For example, a district may have implemented dual credit instruction in its high school, but offered a 
self-paced tutoring program in its middle school. 
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Implementation challenges and supports. Principals and R-Tech facilitators indicated that most 
implementation challenges resulted from the need to clearly communicate program goals to parents and 
staff, as well as from insufficient planning time and from program reporting requirements. Many 
principals and R-Tech facilitators also noted the challenges of implementing a technology-based program 
in districts with outdated computer hardware and insufficient infrastructure to support expanded 
technology resources. Principals and program facilitators reported that strong administrative support, the 
additional revenue provided through the grant, as well as staff buy-in were factors that contributed to 
successful implementation. 

Research Question 2: What is the Level of Student Participation in R-Tech? 

Across Cycle 1 districts, most students were identified for R-Tech services because of weak academic 
performance, including poor Texas Assessment of Knowledge and Skills (TAKS) scores, failing grades, 
and prior academic failure. The number of students participating in R-Tech increased across the 
program’s first year as districts implemented their programs more fully. While less than half of grantee 
districts (47%) offered R-Tech as part of the 2008 su mm er session, nearly all districts (92%) had 
implemented the program for students in spring 2009. Comparisons of the level of participation in R-Tech 
between students receiving services in summer school and students receiving services as part of the 
regular - school year (i.e., fall 2008 and spring 2009) suggest differences in how resources may be used 
during the regular school year and su mm er school. 

Regular school year vs. summer school. In fall 2008, 8,795 students accessed R-Tech resources (an 
average of 97 students per campus) and used resources for an average of 3.7 hours a week. In spring 
2009, 12,736 students accessed R-Tech resources (an average of 129 students per campus) for an average 
of 3.8 hours per week. There were few differences between the characteristics of students who received 
R-Tech services and those who did not during the regular school year. That is, R-Tech students largely 
mi rrored the overall student population in their districts in terms of grade levels served and demographic 
characteristics. The 1,370 students who participated in R-Tech during the 2008 summer session (an 
average of 37 students per campus) had much higher levels of usage than students using R-Tech during 
the regular school year. On average, summer school students accessed R-Tech resources for 8.5 hours 
each week — more than twice the average usage in fall 2008 and spring 2009 — and were more likely to be 
middle school students, with the largest proportion of students (29%) enrolled in the eighth grade. 

Relative to non-participating students, students participating in R-Tech during summer school were more 
likely to be from low income (55% vs. 46%) and minority (50% vs. 36%) backgrounds. These differences 
suggest that some districts implemented R-Tech as an intensive summer school program designed to 
support at-risk middle school students with the transition to high school and to reduce middle school 
retention rates. 

What students study using R-Tech resources. Results from district student usage data indicate that the 
largest proportions of students used R-Tech resources to focus on math (70%) or English/language arts 
(ELA) (46%). Surveyed middle school students were more likely to concentrate on math instruction than 
high school students (42% vs. 26%), and high school students were more likely to focus on ELA (28% vs. 
21%). However, high school students participating in dual credit courses were notably more likely to 
focus on social studies than other R-Tech students in Grades 1 1 and 12 (60% vs. 10%). 

Barriers to student participation in R-Tech. Student resistance, conflicts with extra-curricular 
activities, and transportation challenges limited students’ ability to participate in R-Tech. To address 
barriers, districts expanded R-Tech access times, required participation for some students, and 
implemented incentives to student participation (e.g., offering snacks). Students also reported that slow 
computers, weak school infrastructure, software that was poorly matched to students’ instructional needs, 
and teachers’ lack of technical skills created challenges to participation. 
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Benefits of student participation. Staff on R-Tech campuses and students who received services during 
the program’s first year reported a range of benefits from participation in the grant. Teachers indicated 
that R-Tech had improved students’ academic outcomes, noting that grades had improved and that 
students who recovered credits were able to progress to the next grade on time. Teachers and students 
reported that participation in R-Tech had improved the confidence of some students and that self-paced 
programs eliminated the pressure students felt to keep up with the pace of classroom instruction. In 
addition to academic benefits, students appreciated the convenience of using technology for learning and 
the expanded access to information offered by online resources. Students also felt that their improved 
proficiency using computers would benefit them in college and the workplace. 

Research Question 3: What is the Effect of R-Tech on Teachers? 

In grant applications, all Cycle 1 districts indicated that R-Tech resources would be used to expand 
teachers’ access to technology-based professional development activities; however, results from teacher 
surveys and focus group discussions suggest that many teachers were unaware of the R-Tech resources 
available to them and that few teachers participated in R-Tech professional development opportunities 
during the grant’s first year. 

R-Tech professional development. About 38% of teachers responding to the spring 2009 survey 
participated in training offered as paid of R-Tech. Most teachers reported that training addressed 
preparation for standardized tests, using technology to provide instruction, working with at-risk students, 
and topics related to the use of new computer hardware and software. Across training topics, less than a 
quarter of surveyed teachers reported training was technology-based. Instead, most teachers reported that 
training was provided in face-to-face formats, such as workshops. District-provided data on teacher use of 
online training resources indicate that about 800 teachers (approximately 22% of all teachers working on 
R-Tech campuses) accessed online training opportunities during the 2008-09 school year, spending about 
16 hours, on average, using online training resources, and that middle school teachers had higher average 
rates of usage (19 hours) than high school teachers (16 hours). 

Other opportunities provided by R-Tech. Beyond professional development opportunities, teachers 
reported that they benefitted from the increased access to technology provided by R-Tech, noting that 
improvements to computer labs enabled them to create lessons that integrated technology. Teachers also 
appreciated that R-Tech resources facilitated the development of differentiated lessons and increased 
students’ engagement in learning. Teachers also noted that R-Tech resources had been underused in the 
program’s first year. Some principals reported that information about R-Tech had not been fully 
communicated to teachers and that they would take steps to encourage greater teacher use during the 
program’ s second year. 

Research Question 4: What is the Effect of R-Tech on Student Outcomes? 

The sections that follow present results from analyses of R-Tech on students’ TAKS outcomes. However, 
test results are a limited indicator of R-Tech program effects because most standardized tests lack the 
sensitivity needed to measure incremental increases in student achievement produced by supplemental 
programs such as R-Tech. Given this limitation, readers are asked to consider this report’s findings as 
preliminary. The evaluation’s final report (fall 2010) will include a broader range of student outcome 
data, including graduation and attendance rates, advanced course completions, and indicators of college 
readiness, that were not available at the time of this report’s writing. 

The effect of access time. Students who spent more time using R-Tech resources did not experience 
improved testing outcomes relative to students who spent less time with resources. However, results 
should be interpreted with caution because researchers were not able to control for unobserved student 
differences that may have affected outcomes. For example, students who spent more time using R-Tech 


IV 



resources may have been at greater academic risk, requiring more remediation time than students who 
used R-Tech for briefer periods. If this was the case, then the lack of effect for time spent accessing 
R-Tech may reflect the characteristics of the students identified for more intensive support rather than the 
effects of the support itself. 

Program type. The small number of districts offering one-to-one tutoring with online instructional 
support, technology immersion programs, and iPods loaded with instructional content prevented their 
inclusion in the statistical analysis of program type; therefore, analyses were limited to students 
participating in self-paced programs and dual credit courses. Students participating in self-paced 
programs experienced reduced TAKS scores in reading/ELA relative to R-Tech students who participated 
in other program types; however, self-paced programs had no effect on TAKS outcomes in mathematics, 
science, and social studies. Again, results should be interpreted with caution because it was not possible 
to control for the student characteristics that may have caused students to be identified for self-paced 
programs. If students identified for self-paced programs had more serious academic deficiencies than 
students identified for other types of R-Tech programs, then results may have been produced by 
unobserved student characteristics rather than program participation. 

Supplemental vs. non-supplemental instruction. Students who received R-Tech services as 
supplemental instruction offered outside of the regular school day experienced reduced TAKS testing 
outcomes in social studies relative to students who participated in R-Tech as part of the regular - school 
day (i.e., non-supplemental programs). The effects of supplemental programs on students’ reading/ELA, 
science, and mathematics were persistently negative, but not by statistically significant levels. These 
findings suggest that R-Tech services implemented as part of regular instruction may improve students’ 
TAKS outcomes; however, the characteristics of students identified for supplemental services may have 
affected outcomes. That is, students identified for supplemental services may have struggled 
academically, while students participated in non-supplemental services irrespective of academic need, 
which may indicate that testing outcomes reflect the effects of students’ academic characteristics rather 
than program participation. 

Research Question 5: How Cost Effective is R-Tech? 

Similar - to findings for R-Tech’ s effects on student achievement, readers are asked to consider results of 
this report’s cost-effectiveness analysis as preliminary. Districts varied in the degree to which they 
accessed grant funding over R-Tech’ s first implementation year - . While some districts accessed nearly all 
of their state grant funding during R-Tech’ s first year - (May 2008-May 2009), other districts used little or 
no state funding. This limitation will be offset in the final evaluation report (fall 2010), which will include 
data from the full 2-year - grant period when districts will have accessed nearly all of their funding. Note 
that findings on R-Tech’ s cost effectiveness are limited to districts’ use of state grant funding and do not 
include information on districts’ use of matching funds. 

The allocation of R-Tech funding. Districts report their expenditures of state grant funding through 
TEA’s Expenditure Reporting (ER) system, which includes five spending categories: (1) payroll costs, (2) 
professional and contracted services, (3) supplies and materials, (4) other operating costs, and (5) capital 
outlay. Program budgets included in grant applications indicated that most R-Tech districts characterized 
purchases of computer hardware as “supplies and materials,” but some districts included computer 
hardware in “capital outlay.” Further, most districts characterized computer software as “professional and 
contracted services,” but others included software as “supplies and materials,” or “capital outlay.” 
Variations in how districts budgeted computer hardware and software make it difficult to clearly identify 
these expenditures in the ER system data. 

Acknowledging this limitation, analysis of R-Tech expenditures indicates that most districts invested 
heavily in computer hardware and software during the program’s first year. The largest share of grant 



funding (67%) was allocated to “supplies and materials” and about 10% of funding was spent on “capital 
outlay.” In grant budgets, districts indicated purchases of laptop and desktop computers, LCD projectors, 
printers, furniture for computer labs, and instructional software in both expenditure categories. 

Districts implementing self-paced and technology immersion programs spent more on “supplies and 
materials” and “capital outlay,” as did districts that implemented R-Tech as paid of the regular school day 
(i.e., non-supplemental programs). While districts' average first-year expenditures on “supplies and 
materials” and “capital outlay” were $29,338 and $4,378, respectively, districts implementing self-paced 
programs spent about $29,830 on “supplies and materials” and about $4,443 on “capital outlay.” Districts 
implementing technology immersion programs spent about $67,650 on “supplies and materials” and did 
not allocate funds for “capital outlay.” 4 Districts implementing non-supplemental programs spent about 
$36,890 on “supplies and materials” and $6,625 on “capital outlay.” 

About 15% of state grant funding was spent on “professional and contracted services” during R-Tech’s 
first year. Expenditures in this category included tuition and fees for dual credit courses and payments for 
professional development, technical support services, and educational software. Districts implementing 
dual credit and distance learning courses and one-to-one tutoring and online support spent more in this 
category. Only 8% of first year grant funding was spent on “payroll costs.” Payroll expenditures covered 
the costs of salaries for newly hired computer lab facilitators, extra-duty pay for teachers who worked 
before or after school to provide R-Tech services, and the costs of substitutes to enable teachers to 
participate in professional development. Districts did not spend any state funding for “other operating 
costs.” 

The cost effectiveness of program configurations. In spite of substantial start up costs in terms of 
investments in technology resources, districts that implemented R-Tech for larger numbers of students 
experienced the lowest per-student program costs. Across Cycle 1 districts, the average per-student cost 
of providing R-Tech services during the program’s first year was $420. Districts that implemented 
programs serving 500 or more students experienced average per-student costs of $1 1 1, while districts that 
served fewer than 50 students during R-Tech’s first year had average per-student costs of more than 
$1,500. R-Tech districts that implemented self-paced programs had average per-student costs that were 
slightly above average ($428) and districts implementing dual credit and distance learning programs had 
per-student costs that well below average ($198). This difference is likely the result of greater investment 
in technology resources needed to implement self-paced programs. Although technology immersion 
programs spent heavily on computer resources during R-Tech’s first year, districts implementing this type 
of program experienced below average per-student costs ($269) because large numbers of students 
participated in the program. Districts that implemented R-Tech using iPods loaded with instructional 
content served fewer students and had average per-student costs of about $358. 5 Across program 
configurations, per-student implementation costs are expected to drop during R-Tech’s second year as 
more students gain access to resources purchased in the grant’s first year. 

Supplemental vs. non-supplemental instruction. Districts that implemented R-Tech as paid of regular 
classroom instruction (i.e., non-supplemental programs) experienced substantially lower per-student costs 
than supplemental programs ($182 vs. $612, on average). The difference in costs results from differences 
in the numbers of students served. Districts implementing supplemental programs served an average of 


4 The notably high expenditures for “supplies in materials” is the result of one technology immersion district 
budgeting its full grant award ($200,000) to purchase laptop computers for students and teachers. The district 
accessed 60% of its grant award ($120,886) during R-Tech’s first implementation year. 

5 Neither district offering R-Tech as one-to-one tutoring with online instructional support served students during the 
program's first implementation year. Therefore, it was not possible to identify a per-student cost for this type of 
program. 
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172 students during R-Tech’s first year, while districts implementing non-supplemental programs served 
an average of 350 students. 

Sustainability. Nearly half (48%) of principals responding to the spring survey reported that insufficient 
financial resources created a moderate or substantial barrier to continuing R-Tech after grant funds expire 
in May 2010. Most principals (55%) indicated that R-Tech would be offered as part of classroom 
instruction rather than as a supplemental program at the conclusion of the grant. During interviews 
conducted as paid of spring site visits, several principals said they would only continue R-Tech after the 
grant period if the program demonstrated positive effects on students’ TAKS scores. 

THE ONGOING EVALUATION 

The findings presented in this report arc preliminary and are drawn from R-Tech’s first implementation 
year in Cycle 1 districts. The ongoing evaluation will continue to collect information about how Cycle 1 
districts implement R-Tech, the challenges and benefits of implementation, and the program’s effect on 
student and teacher outcomes, as well as its cost effectiveness across the grant’s second year. More 
conclusive findings for the grant’s full 2-year implementation period will be presented in the final 
evaluation report (fall 2010). 

As discussed earlier in this summary, the final report will include a broader range of student achievement 
indicators and complete information on Cycle 1 districts’ use of state grant funds. The final report will 
also include findings from surveys of R-Tech facilitators, principals and teachers on R-Tech campuses, 
and students who participated in R-Tech services administered in spring 2010, as well as information 
collected during site visits to R-Tech districts in spring 2010. The inclusion of survey and site visit data 
gathered at the grant’s conclusion will enable researchers to identify modifications to districts’ 
implementation plans, changes in respondents’ roles in implementation and perceptions of grant services, 
and how changes may affect student and teacher outcomes. 
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Chapter 1 
Introduction 


The puipose of the evaluation of the Texas Rural Technology (R-Tech) Pilot Program is to assess the 
program’s effectiveness. To that end, the evaluation focuses on how R-Tech is implemented, its effects on 
teacher and student outcomes, as well as its cost effectiveness and sustainability. To address these goals, 
evaluation activities are guided by the following research questions: 

1. How is R-Tech implemented across grantee districts and schools? 

2. What is the level of student participation in R-Tech? 

3. What is the effect of R-Tech on teachers? 

4. What is the effect of R-Tech on student outcomes? 

5. How cost effective is R-Tech? 

The R-Tech evaluation spans the 2008-09 and 2009-10school years and will produce two interim reports 
(fall 2008 and winter 2010), as well as a final report (fall 2010). The evaluation’s first interim (December 
2008) presented descriptive information relevant to Research Question 1. It described the characteristics 
of R-Tech districts and campuses, the students they enroll, and the teachers they employ. In addition, the 
first interim report provided baseline 1 information about students’ academic outcomes and described 
districts’ implementation plans as outlined in their grant applications to the Texas Education Agency 
(TEA). The findings presented here comprise the evaluation’s second interim report and expand upon the 
first interim report to present information relevant to each of the evaluation’s research questions based on 
data collected across R-Tech’ s first implementation year (May 2008- May 2009), including the 2008 
summer session, as well as the fall 2008 and spring 2009 semesters. 

This chapter provides an overview of the R-Tech program, as well as background information on the 
challenges faced by rural schools and the potential of technology to overcome these challenges. It also 
discusses the methodological issues inherent in evaluating the effects of supplemental educational 
interventions, such as R-Tech, and it introduces the methodologies and data sources that produce the 
second interim report’s findings. The chapter concludes with a discussion of the report’s limitations and 
an overview of the ongoing evaluation. 

OVERVIEW OF R-TECH 

In 2007, the 80th Texas Legislature authorized the creation of a pilot program designed to provide 
technology-based supplemental educational services to the state’s rural districts. House Bill (HB) 2864 
(80th Texas Legislature, Regular Session, 2007) authorized TEA to create R-Tech, which provides nearly 
$8 million to be used to support technology-based supplemental education programs, including online 
courses, in the state’s rural districts from May 2008 through May 2010. In order to be eligible for R-Tech 
funding, districts must have enrolled fewer than 5,000 students, and must not have been located in a 
metropolitan area as of January 1, 2007. Priority in grant awards was given to districts with limited course 
offerings and to districts with high academic need as demonstrated by their 2007 accountability ratings. 
Grantee districts receive $200 per school year in state grant funding for each student receiving R-Tech 
services and are required to provide $100 per participating student per school year in matching funds. 2 As 
a condition of funding, districts are expected to provide students with access to R-Tech services for a 
minimum of 10 hours per week. 


'Baseline indicators are measures of school characteristics and performance prior to program implementation. Such 
measures provide a “baseline” from which to assess program effects. 

“Districts may use High School Allotment (HSA) monies to provide matching funds at the high school level. 
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R-Tech funding was awarded in two grant periods, or cycles. Cycle 1 of the grant awarded about $6.3 
million in funding to be used during the May 1, 2008, through May 31, 2010, project period. In the spring 
of 2008, TEA awarded 64 districts Cycle 1 funding. Over the course of the 2008-09 school year, one 
Cycle 1 grantee district opted not to participate in the program, which reduced the total number of Cycle 1 
districts to 63. 3 In fall 2008, TEA awarded about $1.5 million in R-Tech Cycle 2 funding to 19 districts. 4 
Cycle 2 funding must be used during the January 1, 2009, through May 31, 2010, project period. R-Tech 
funding supports technology-based supplemental education services to students in Grades 6 through 12. 
Such services may include: 

• Research-based instructional support, 

• Teacher training, 

• Academic tutoring or counseling, 

• Distance learning opportunities in the core content areas or in foreign languages, and 

• Dual credit coursework in the core content areas or in foreign languages. 

RURAL SCHOOLS AND TECHNOLOGY 

Although education policy and reform discussions have tended to focus on the problems of urban districts 
and inner-city students, rural schools, and the students who attend them, confront a range of challenges 
resulting from social and geographic isolation, inadequate school and community resources, as well as 
declining enrollments (Johnson & Strange, 2007). In 2003-04, more than half of the nation’s school 
districts and a third of its public schools were located in rural areas, but rural schools enrolled only 20% 
of the nation’s public school students (National Center for Education Statistics [NCES], 2007, p. iii). As 
these statistics suggest, rural schools tend to enroll fewer students, on average, than their counterparts in 
other locales, and receive less in terms of overall per-pupil funding (Johnson & Strange, 2007). Beyond 
funding disparities, the geographic isolation of rural communities makes it difficult to recruit and retain 
high quality teachers, and teacher shortages prompt many rural districts to rely on unqualified or out-of- 
field teachers in hard to staff courses (Hobbs, 2004; Jimerson, 2003, 2004; Lemke, 1994; Stern, 1994). In 
the absence of qualified teachers, many rural districts struggle to provide a comprehensive curriculum, 
particularly at the high school level, and to provide supplemental educational support to students who 
need remediation, tutoring, and other services designed to increase academic achievement (Hobbs, 2004; 
Jimerson, 2003). 

Texas enrolls more students in rural public schools than any other state (Johnson & Strange, 2007). 
Eighteen percent of Texas’ more than 4.5 million public school students attend a rural public school, and 
more than half of the state’s public school districts and 27% of its public schools are located in rural areas 
(authors’ calculations using NCES and TEA data, 2007). Relative to non-rural Texas schools and rural 
schools nationally, Texas’ rural schools serve larger proportions of English language learners (ELL) and 
students from low income backgrounds (Jimerson, 2004; Johnson & Strange, 2007). Texas’ rural districts 
have higher average rates of teacher turnover and a greater incidence of out-of-field teaching relative to 
the state’s non-rural districts or rural schools nationally (Jimerson, 2004). In addition, Texas’ rural 
districts lag the state’s non-rural districts and rural districts nationally in providing opportunities for 
students to participate in supplemental programs focused on enrichment or remediation (Jimerson, 2004). 

Technology is increasingly recognized as a cost-effective means to overcoming the challenges faced by 
rural schools (Malhoit, 2005). Through the use of technology, rural schools may offer students “an 
advanced, varied, and cost-effective curriculum” by providing access to online courses and distance 
learning opportunities (Malhoit, 2005, p. 20). In addition to increasing academic rigor and the diversity of 
course offerings available to rural students, technology also holds the potential to provide supplemental 


3 The district that chose to withdraw from R-Tech did not access any of its grant award. 

4 Three of the 19 Cycle 2 districts also received Cycle 1 grant awards. 
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programs, such as online tutoring and remediation, for rural students who struggle academically (Griffin, 
2005; Malhoit, 2005). Rural students may also benefit when their teachers participate in online 
professional development and training designed to improve instruction and classroom management skills 
(Cullen, Frey, Flinshaw, & Warren, 2004, 2006; Flobbs, 2004; Wright & Lesisko, 2008). 

Despite the potential benefits of technology, many rural schools are unprepared to use technology to 
enhance teaching and learning. A study of school facilities found that many rural districts failed to 
adequately maintain their buildings, and that long-term underinvestment in school buildings threatened 
the ability of rural districts to implement new systems of technology and to accommodate new approaches 
to instruction (Dewees & Hammer, 2000). In addition, many rural schools lack the infrastructure and 
resources needed to adequately implement programs that rely on technology-based instruction. 

Nationally, 50% of rural schools have outdated wiring that will not support broad technology use, 84% 
lack fiber optic cable, and 46% do not have operational computer networks (McColl & Malhoit, 2004, 
p. 5). Further, the long-term success of technology-based interventions depends on the sustainability of 
technology. Grant revenue may be sufficient to get projects off the ground, but in order to see long term 
gains, districts must design plans that arc sustainable when grant funding expires (Mason, Smith, & Gohs, 
1982). Sustaining technology-based initiatives may be particularly challenging for small, rural districts 
with low enrollments and inadequate tax bases (Dewees & Hammer, 2000). 

Recognizing the potential of technology to expand opportunities for students and teachers in isolated, 
rural areas, federal- and state-level policymakers have introduced a variety of programs to assist rural 
schools in obtaining the infrastructure, technology hardware and software, and training needed to 
effectively implement technology into instructional practice. R-Tech is one of several Texas programs 
designed to improve access to technology resources and technology-based instruction in low income and 
underserved districts (e.g., the Technology Integration in Education Initiative, the Texas Technology 
Immersion Pilot, Vision 2020 Grants). R-Tech is somewhat unique in that it is targeted specifically to 
Texas’ small, rural districts, and for its focus on the provision of technology-based supplemental 
instruction. 

SUPPLEMENTAL PROGRAMS 

Supplemental programs, such as R-Tech, are generally offered outside of a student’s regularly scheduled 
classes, often before or after school, and are designed to provide additional instructional support for 
struggling students. While tutoring before or after school has been a longstanding feature of most 
educational systems, the provision of formalized supplemental education services, or SES, has gained 
fraction in recent years in response to the No Child Left Behind (NCLB) Act’s requirement that low 
income students attending persistently low-performing schools receive access to free tutoring in math and 
reading. Although R-Tech operates outside of NCLB’s parameters for the provision of SES, the grant’s 
preference for districts with weak accountability ratings reflects NCLB’s reasoning that supplemental 
instruction will improve academic outcomes in poor-performing schools. To date, however, there is little 
empirical evidence to support this thinking (Burch, Steinberg, & Donovan, 2007; Munoz, Potter, & Ross, 
2008; Ross, Paek, & McKay, 2008). 

Challenges to Evaluating the Effectiveness of Supplemental Programs 

The lack of empirical evidence for the effectiveness of supplemental instruction may indicate that such 
programs do not have a measurable effect on student learning, but it is also possible that variations in SES 
providers and the types of services they provide make it difficult to identify what the actual “effect” of 
services may be. The effects of SES also may be difficult to isolate from the range of other influences that 
affect student learning during the school year, and the non-random assignment of students to SES makes 
it difficult to know whether observed effects result from participation in SES or the characteristics of the 
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students who receive services. Each of these challenges to evaluating SES is discussed in greater detail in 
the sections that follow. 

Many SES providers. In part, the lack of evidence of the effectiveness of SES is due to wide variation in 
the type and quality of services provided to students. That is, SES is not a single intervention 
implemented in a uniform manner across schools and districts. According to Ascher (2006), 
approximately 2,000 approved vendors provided NCLB-required SES in one or more states during the 
2005-06 school year. The broad range of vendors providing supplemental instruction has made it difficult 
to arrive at an overall estimate of program effectiveness, although studies of specific SES providers have 
produced mixed results. For example, a 2-year study of multiple vendors providing SES in Tennessee 
found statistically significant effects 5 on student outcomes for only two vendors, both of which were 
negative (Potter, Ross, Paek, McKay, Sanders, & Ashton, 2008). A 3-year evaluation of more than 40 
vendors providing supplemental instrucion in the Chicago public school system found wide variations in 
individual vendor effects (both positive and negative), and concluded that, overall, the district’s program 
had a small, positive effect on student reading outcomes, but a negligible effect on math outcomes 
(Chicago Public Schools, Office of Research, Evaluation, & Accountability, 2007). A summary of the 
findings of several state-level evaluations that included more than 200 SES vendors found only four 
vendors that produced statistically significant positive effects on student outcomes, although 57% 
produced some margin of positive learning growth (Ross, Paek, & McKay, 2008, p. 30). 

Difficulty isolating the effects of SES. In addition to the problem of multiple vendors, researchers 
struggle to distill the effects of supplemental education from the wide range of other factors that influence 
student learning. Experiences in core content area classrooms, teacher quality, student motivation and 
interest, and a range of extraneous variables affect educational outcomes, making it difficult for 
researchers to isolate the effects of supplemental programs on achievement (Munoz, Potter, & Ross, 

2008; Ross, Paek, & McKay, 2008). Some researchers also have noted that students receive supplemental 
services for a relatively small proportion of the school year, and most standardized tests, such as the 
Texas Assessment of Knowledge and Skills (TAKS), lack the sensitivity to measure incremental changes 
in achievement (Baker, 2007; Linn & Miller, 2005; Kane, 2004). 

Nonrandom assignment of students to SES. A further challenge to the evaluation of supplemental 
programs arises because students are not randomly assigned to participate in services. Students choose to 
participate in SES or are assigned to receive services, and differences in the characteristics of the students 
who receive services relative to those who do not make comparisons difficult. For example, if SES 
participants are more motivated students who are willing to come before or after school to receive 
tutoring, then differences in the test scores between SES and non-SES participating students may reflect 
differences in students’ motivational levels rather than the effects of services. It is not possible for 
researchers to observe and quantify the many characteristics that affect student participation in 
supplemental instrucions, and the influence of unobserved traits may distort estimates of program 
effectiveness. 

Given the methodological challenges to assessing the effects of supplemental programs on student 
achievement outcomes, Ross, Paek, and McKay (2008) suggest that researchers take a larger view of the 
effects of SES and consider qualitative outcomes such as improved student self-esteem, motivation, and 
study skills, as well as test scores. The authors warn: 


5 An outcome is said to be “statistically significant” when the probability of the outcome occurring (e.g., increased 
reading TAKS scores) by chance is less than an established level of probability. For example, if a program increased 
TAKS reading scores by a statistically significant amount at the 5% level of probability, then the likelihood of that 
outcome occurring by chance alone is only 5%. 
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To the extent the evaluation studies and the public weigh SES on the basis of immediate 
achievement gains only, we could well end up with the possibly misleading “black-or-white” 
conclusion that SES is ineffective and needs to be discontinued (like so many other educational 
programs in the past) (p. 31). 

REPORT METHODOLOGY 

These concerns guide the approach to evaluating the supplemental services provided to Texas’ rural 
districts through R-Tech. In order to avoid misleading conclusions drawn from focusing solely on student 
testing outcomes, the findings presented in the second interim report arc drawn from multiple data sources 
and rely on a variety of quantitative and qualitative methodologies. The sections that follow describe the 
report’s data sources and approach to analyses. 

Document Analyses 

In order to gain a fundamental understanding of the types of programs implemented in R-Tech districts, 
district goals for the program, and districts’ planned use of grant funds, researchers analyzed grantee 
districts’ grant applications and progress reports to TEA. Analysis of district grant applications enabled 
researchers to categorize R-Tech programs by type (e.g., self-paced programs and dual credit/distance 
learning) and to understand how districts planned to use grant funds to support program goals (e.g., the 
purchase of laptops or software, salaries for computer lab staff). Examination of districts’ progress reports 
allowed researchers to identify changes in districts’ implementation strategies across R-Tech’s first year 
and to refine program categories. 

Analysis of R-Tech applications and progress reports identified the following five categories of R-Tech 
programs: 

1. Self-paced software, including remediation, online tutoring, and credit recovery programs; 

2. Dual credit and distance learning programs that enable students to receive credit for college 
courses that also meet high school requirements; 

3. One-to-one tutoring with online instructional support; 

4. School-wide technology immersion programs in which every student receives a laptop computer; 
and 

5. Programs that incorporate iPods to deliver tailored instructional content to specific student 
groups. 

The categories are not discrete across R-Tech districts. That is, districts may offer more than one type of 
program. For example, a district may offer dual credit and distance learning opportunities at its high 
school and a self-paced tutoring program at its middle school. Each program category is discussed in 
more detail in chapter 3. 

Analysis of districts’ progress reports also revealed that many Cycle 1 districts (40%) implemented 
R-Tech as paid of classroom instruction (i.e., non-supplemental programs). For example, some districts 
used R-Tech funding to purchase laptop computers and software that teachers incorporated as paid of 
daily lessons. Some districts updated and expanded computer labs, and teachers scheduled class time in 
the lab to access R-Tech resources, or assigned groups of students to the lab during class as a means of 
differentiating instruction. Recognizing that districts implementing R-Tech as a supplemental 
instructional program may have different outcomes than districts implementing R-Tech as paid of regular 
instruction, researchers also categorized programs as supplemental or non-supplemental based on when 
students accessed R-Tech services. 
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These categories are used in the report’s quantitative analyses to identify whether different 
implementation strategies and whether R-Tech is provided as a supplemental or non-supplemental 
program have varying effects on student outcomes and implementation costs. Because the categories used 
in this report are based solely on document analysis, researchers advise that the categories are preliminary 
evaluation tools. The ongoing evaluation will include program categories on subsequent surveys of 
principals and facilitators and ask respondents to identify the program type that best describes their 
schools’ approach to R-Tech implementation. The inclusion of program descriptors on surveys will 
enable researchers to gain a more refined understanding of variations in districts’ implementation 
strategies and to assess more fully how different approaches to implementing R-Tech may affect student 
outcomes and program costs. 

Quantitative Data Sources 

The evaluation incorporates quantitative data drawn from archival sources, such as Texas’ Public 
Education Information Management System (PEIMS) and Academic Excellence Indicator System 
(AEIS), as well as data collected directly from R-Tech districts. 

Archival data. PEIMS is an archival database that contains data collected from Texas public schools by 
TEA. PEIMS includes student demographic and academic performance data, as well as information about 
school staffing, finance, and organization. AEIS is an archival database that contains information about 
the academic performance and accountability rating of each public school district and campus in Texas. 
Some analyses also incorporate data included in TEA’s public school directory, known as AskTED. 

District-provided data. Districts awarded R-Tech grants are required to track the average number of 
hours per week that individual students participate in services provided by R-Tech (TEA, 2008b), and 
many districts selected software packages and vendors that facilitated the collection of student usage data. 
In addition, districts collected data about teachers’ participation in professional development activities 
with the use of R-Tech funds. Student and teacher usage data were provided to TEA through a data 
upload system hosted by the Agency. Districts submitted upload data for students and teachers 
participating in R-Tech at three points across the project’s first implementation year: (1) the conclusion of 
the 2008 summer session, (2) the conclusion of the fall 2008 semester, and (3) the conclusion of the 
spring 2009 semester. Districts also provided information on their use of R-Tech grant funds through 
TEA’s Expenditure Reporting (ER) system. 

Quantitative Analyses 

Descriptive statistics. Analyses included in chapter 2 rely on PEIMS and AEIS data to present 
descriptive information about R-Tech districts and campuses, as well as the students they enroll. The 
academic outcomes of R-Tech campuses are compared to statewide averages of campuses serving 
roughly the same grade levels (i.e., Grades 6 through 12), as well as to TEA-identified peer comparison 
campuses that serve similar student populations, where appropriate. 

Regression analyses. The effect of R-Tech services on students’ 2009 TAKS testing outcomes is 
estimated using PEIMS and student upload data and hierarchical linear modeling (HLM) regression 
methods, which allow researchers to control for student- and campus-level characteristics. A more 
detailed discussion of the approach to estimating the effect of R-Tech on student outcomes and estimation 
results is presented in chapter 6 and Appendix F. 

Cost-effectiveness analysis. The cost effectiveness analysis presented in chapter 7 provides preliminary 
information about how R-Tech districts allocated state grant funding over the course of the pilot’s first 
year. Analyses include calculations of the per-student cost of implementing R-Tech, recognizing wide 


6 



variations in district levels of implementation across R-Tech’s first year, and discuss the project’s 
sustainability once grant funds expire. 

Surveys 

This evaluation report incorporates the results of three online surveys: (1) a survey of principals of 
R-Tech campuses and R-Tech facilitators, (2) a survey of teachers working on R-Tech campuses, and (3) 
a survey of students receiving R-Tech services. Results from surveys arc presented in chapters describing 
R-Tech implementation in 2008-09 (chapter 3), chapters discussing R-Tech’s effects on students and 
teachers (chapters 4 and 5), and the sustainability of R-Tech services (chapter 7). An overview of each 
survey is presented in the sections that follow. 

Online survey of R-Tech facilitators and principals of R-Tech campuses. A voluntary, online survey 
of R-Tech facilitators and principals was administered in fall 2008 and again in spring 2009. The fall 
2008 survey measured respondents’ initial understanding of the R-Tech pilot and the early challenges in 
implementing the program. The spring 2009 survey measured changes in respondents’ perceptions of 
R-Tech across the project’s first year, the ongoing challenges to implementation, and the approaches by 
which districts overcame challenges. The survey also probed the effects of R-Tech on teachers and 
students, as well as principals’ views of the sustainability of R-Tech services after grant funds expire. A 
detailed description of survey administration procedures, survey response rates, characteristics of survey 
respondents, supplemental tables cited in report chapters, and copies of the fall and spring surveys are 
included in Appendix A. 

Online survey of teachers on R-Tech campuses. Similar to the survey of R-Tech facilitators and 
principals, a voluntary, online survey of teachers on R-Tech campuses was administered twice during the 
2008-09 school year — once in fall 2008 and again in spring 2009. The fall 2008 survey asked about 
teachers’ roles in planning and implementing R-Tech, the professional development opportunities 
provided as part of R-Tech, the program’s effect on teachers, and teachers’ overall understanding of the 
goals of R-Tech. The spring 2009 survey measured changes in teachers’ perceptions of R-Tech and their 
role in implementation across the 2008-09 school year. The teacher survey, a detailed description of 
survey administration procedures, survey response rates, and supplemental tables referenced in report 
chapters are included in Appendix B. 

Online survey of students participating in R-Tech. In spring 2009, students who participated in R-Tech 
services either in su mm er 2008 or during the 2008-09 school year were invited to participate in a 
voluntary, online survey that asked about students’ access to R-Tech technology resources, their views of 
technology-based instruction, as well as what students liked most and least about learning with 
technology. The student survey, a detailed description of survey administration procedures, and 
respondent characteristics are included in Appendix C. 

Qualitative Data and Analysis: Site Visits to R-Tech Districts 

In order to gain a more nuanced understanding of the ways in which districts implement R-Tech services, 
the challenges and benefits of implementation, and R-Tech’s effects on students and teachers, researchers 
conducted site visits to eight R-Tech districts (13 campuses) in spring 2009. Site visits included 
interviews with campus administrators, R-Tech facilitators, focus group discussions with teachers 
involved in R-Tech, and focus group discussions with students who received R-Tech services, as well as 
observations of R-Tech service delivery. Districts selected for site visits differed in the types of R-Tech 
programs implemented, but had programs that were largely representative of the range of R-Tech 
programs offered across all 63 Cycle 1 grantee districts. For example, site visit districts included 
campuses that offered self-paced credit-recovery and tutoring programs, dual credit instruction for high 
school students, programs targeted to specific student populations (e.g., ELLs), programs offered to all 
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students, programs addressing academic goals, and programs providing interventions for students with 
behavioral problems (e.g., truancy, disciplinary referrals). 

Following site visits, researchers reviewed audio files and notes, transcribed interview and focus group 
discussions, and identified response categories and themes for analysis. Transcribed interviews and focus 
groups were analyzed using tables and summaries organized by respondent groups (e.g., principals and 
teachers) and response content. Across interviews and focus groups, analyses focused on how R-Tech was 
implemented in site visit districts, the challenges to implementation, how challenges were overcome, the 
effects of the program on students and teachers, and whether R-Tech would be sustained once grant funds 
expire. Data collected through site visits are presented in combination with survey data to provide more 
robust descriptions of R-Tech implementation (chapter 3), the program’s effects on students and teachers 
(chapters 4 and 5), and the sustainability of R-Tech services (chapter 7). A more detailed discussion of the 
approach to identifying site visit districts and site visit activities, as well as a brief overview of each 
district and its R-Tech program are included in Appendix D. 

LIMITATIONS OF THE SECOND INTERIM EVALUATION REPORT 

Although the R-Tech evaluation relies on a diverse set of quantitative and qualitative data sources and 
methodologies, the second interim report experiences some limitations that arise from the availability of 
student outcome data, variations in districts’ implementation strategies, and survey response rates. Each 
limitation is discussed in a section that follows. Further limitations that may affect specific analyses are 
discussed in context in the evaluation chapter that includes the analysis. 

Limited Information on Student Outcomes 

Although the ongoing evaluation will consider a broader range of student outcome measures, including 
advanced course completions, attendance and graduation rates, and indicators of college readiness, at the 
time of the second interim report’s writing (spring 2009), the only student outcome data available 
subsequent to R-Tech’s implementation were students’ spring 2009 TAKS scores. Therefore, the report’s 
discussion of the first year effects of R-Tech on student outcomes is limited to standardized testing 
results, which, as noted earlier in this chapter, may produce misleading conclusions. Considerable 
research has demonstrated that standardized tests are limited indicators of student achievement that must 
be balanced by consideration of broader measures of learning, such as improved student engagement, 
motivation, and interest; increased graduation and attendance rates; as well as improved study skills and 
grades (Archbald & Newman, 1988; Klein, 1971; Koertz, 2002; Ross, Paek, & McKay, 2008; Russell & 
Fliggins, 2003). 

Acknowledging the limitations of standardized tests to fully measure improvements in student 
achievement outcomes, particularly for supplementary instructional programs (Munoz, Potter, & Ross, 
2008; Ross, Paek, & McKay, 2008), readers are advised to consider this report’s discussion of R-Tech’s 
effects on student outcomes (chapter 6) as preliminary. The final report (fall 2010) will present analyses 
of a broader range of student outcome data and provide more conclusive findings about R-Tech’s effects 
on student achievement. 

Uneven First Year Implementation Across R-Tech Districts 

Variations in the levels of R-Tech implementation across Cycle 1 grantee districts also create limitations 
for the second interim report. During R-Tech’s first implementation year (May 2008-May 2009), most 
districts fully implemented R-Tech, providing students with access to services and enabling teachers to 
participate in grant-provided professional development activities. Flowever, some districts experienced 
challenges that delayed their implementation schedules, which meant that students and teachers had 
reduced access, or in some in cases no access, to R-Tech services across the program’s first year - . 



Researchers had limited ability to control for the range of implementation levels across districts, which 
again suggests that readers should consider the second interim report’s findings as preliminary. The final 
evaluation report (fall 2010) will include information from the full 2-year grant period in which all Cycle 
1 grantee districts will have implemented the program to a largely complete degree, and findings will 
provide a more complete view of R-Tech’s effects. 

Survey Response Rates 

The second interim report relies on survey data collected across R-Tech’s first year of implementation, 
and survey response rates varied across survey administrations and respondent groups. Table 1.1 presents 
campus-level and respondent-level survey response rates for each survey administration and respondent 
group. Campus-level response rates represent the percentage of R-Tech campuses that had respondents 
who participated in surveys (N=l 15 for both fall and spring administrations). Respondent-level response 
rates represent the ratio of individuals who participated in each survey to the total number of the 
respondent group expected to participate, expressed as a percentage. For example, researchers estimated 
the number of teachers who worked in R-Tech campuses in 2008-09 using 2007-08 AEIS data as a proxy 
for 2008-09 staffing patterns, 6 and calculated teacher-level response rates as the ratio of the number of 
teachers participating in surveys to the total number of teachers expected to work on R-Tech campuses, 
expressed as a percentage. Because each campus is only expected to have one principal, the campus-level 
and respondent-level response rates for principals are the same. It was not possible to calculate 
respondent-level response rates for R-Tech facilitators because campuses used different approaches to 
facilitating grant implementation — some campuses assigned facilitator duties to one individual, while 
other campuses distributed responsibilities across several individuals. Student-level response rates 
represent the ratio of students who participated in the spring survey to the total number of unique students 
who were included in district upload data submitted to TEA for the su mm er 2008, fall 2008, and spring 
2009 reporting periods. Students included in the data uploads for more than one reporting period were 
counted only once in the calculation of the response rate. Appendices A, B and C provide more 
information on survey administration procedures and response rates, as well as the characteristics of 
survey respondents. 


6 2008 AEIS data were the most current data available at the report's writing. 
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Table 1.1 

R-Tech Campus-Level and Respondent-Level Response Rates, by 
Respondent Group: Fall 2008 and Spring 2009 


Respondent Group 

Fall 2008 

Spring 2009 

Campus-level response rate 3 

Principals 

71.3% 

65.2% 

R-Tech facilitators 

65.2% 

56.5% 

Teachers 

80.0% 

67.0% 

Students 

— 

47.0% 

Respondent-level response rate 

Principals 

71.3% 

65.2% 

R-Tech facilitators 0 

Unknown 

Unknown 

Teachers (estimated) 11 

33.0% 

15.7% 

Students 6 

— 

20.1% 


Sources: R-Tech Principal/Facilitator Survey: fall 2008 and spring 2009; R-Tech 
Teacher Survey: fall 2008 and spring 2009; R-Tech Student Survey: spring 2009; Texas 
Education Agency (TEA) Student Upload data: summer 2008, fall 2008, spring 2009; 

TEA 2008 Academic Excellence Indicator System (AEIS) campus staff statistics file. 

Notes. Principals who serve dual roles — principals and R-Tech facilitators — are 
included in the campus-level response rates for both groups. The student survey was 
only administered in spring 2009. 

‘‘The percentage of R-Tech campuses with survey respondents (N=l 15 for both fall and 
spring). 

b The ratio of survey respondents to the total number of individuals in the respondent 
group, expressed as a percentage. 

“Variations in the number of facilitators per campus across R-Tech districts prevented 
the calculation of a facilitator-level response rate. 

d The teacher-level response rate is calculated as the ratio of teachers responding to the 
survey to the total number of teachers working on R-Tech campuses as reported in the 
2008 TEA AEIS campus statistics data files, expressed as a percentage. 2007-08 data 
are used as a proxy for 2008-09 staffing patterns. 

“The student-level response rate is calculated as the ratio of students participating in the 
spring survey to the total number of unique students included in data uploads to TEA 
for the summer 2008, fall 2008, and spring 2009 reporting periods, expressed as a 
percentage. Students included in multiple uploads were included only once in the 
calculation. 

In spite of weaker response rates to spring 2009 surveys of principals, R-Tech facilitators, and teachers, 
the second interims report’s discussions of survey findings focus more heavily on spring results because 
spring respondents had greater awareness of the R-Tech program and its effects. As noted in the previous 
section, R-Tech implementation varied across the program’s first year, and many districts delayed 
implementing services until the spring 2009 semester. Comparisons of fall and spring survey responses 
indicate that many respondents to the fall survey had little of knowledge of the R-Tech program in their 
districts. In recognition of this limitation, some findings from the fall 2008 surveys are included in 
appendices, and chapter discussions focus on results from spring 2009 surveys. In other cases, however, 
report chapters present parallel findings for the fall and spring surveys as a means to demonstrate changes 
in respondents’ understandings and perceptions of the R-Tech program across its first implementation 
year. 
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THE ONGOING EVALUATION 


The ongoing evaluation will offset many of the limitations discussed in the previous sections. The final 
evaluation report (fall 2010) will expand to include a broader range of student outcome indicators, 
including course completion, attendance, and graduation rates in R-Tech districts, and will provide a more 
complete understanding of R-Tech’ s effects on student achievement. The final evaluation report will also 
present information about districts’ full use of grant funding and will link the use of funding as well as 
program configurations to student outcomes, such as TAKS scores and graduation rates, which will 
enable a more complete understanding of variations in program cost effectiveness. 

The ongoing evaluation will include surveys of R-Tech facilitators, principals and teachers on R-Tech 
campuses, and students receiving R-Tech services administered in spring 2010, when programs are fully 
implemented and respondents have greater awareness of R-Tech and its effects. The spring 2010 surveys 
will provide information about second year implementation of R-Tech, the continued challenges and 
successes of implementation, as well as changes in respondents’ roles in implementation and perceptions 
of program effects. The spring 2010 surveys will also ask principals and R-Tech facilitators to identify the 
type of R-Tech program implemented in their districts using the researcher-identified program categories 
discussed earlier in this chapter, and whether their programs are supplemental or implemented as paid of 
classroom instruction (i.e., non- supplemental). Survey responses will provide more information about the 
types of R-Tech programs offered and allow researchers to more fully assess the effect of program 
configuration on student outcomes and cost effectiveness. 

In addition, the final evaluation report will include information from a second set of site visits to R-Tech 
districts, conducted in spring 2010. Second year site visits will include interviews with principals and 
R-Tech facilitators, focus group discussions with teachers and students receiving R-Tech services, as well 
as observations of R-Tech service delivery. The spring 2010 site visits will provide detailed information 
about how R-Tech implementation may have changed in its second year, and how changes may affect 
program outcomes. 

THE STRUCTURE OF THE SECOND INTERIM EVALUATION REPORT 

The second interim report for the R-Tech evaluation is organized as follows: 

• Chapter 1 provides background on the R-Tech grant, introduces the evaluation’s research 
questions, and reviews the literature on the role of technology-based instruction in improving 
outcomes for rural districts, as well as the challenges to assessing the effects of supplemental 
instruction. The chapter also discusses the report’s data sources, methodologies, and limitations, 
and includes information on the ongoing evaluation. 

• Chapter 2 presents descriptive information about the characteristics of Cycle 1 grantee districts 
and campuses implementing the R-Tech program, including information about staff and students. 

• Chapter 3 discusses first year implementation of R-Tech in Cycle 1 districts. It provides a detailed 
discussion of the types of programs districts implemented during the R-Tech’ s first year, as well 
as stakeholder roles in program implementation. The chapter also discusses the challenges 
districts experienced in implementing R-Tech and strategies for overcoming challenges. 

• Chapter 4 includes information about R-Tech’ s effects on students drawn from student upload 
and PEIMS data, surveys with program stakeholders and site visit interviews and focus group 
discussions, and includes information about the characteristics of students receiving R-Tech 
services, the barriers that may limit student participation in the program, strategics for 
overcoming barriers to participation, and the benefits and challenges of student participation. 

• Chapter 5 presents findings of R-Tech’ s effects on teachers and includes information about 
professional development provided in support of the grant, as well as the effects of R-Tech on 
classroom instruction. 
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Chapter 6 provides preliminary information about R-Tech’s effects on students’ 2009 TAKS 
scores and how differences in districts’ program configurations may affect outcomes. 

Chapter 7 presents preliminary information about how R-Tech districts allocate state grant 
funding and how different implementation strategies may influence the program’s cost 
effectiveness. 

Chapter 8 presents a summary of findings relative to each of the evaluation’s research questions, 
and discusses the policy implications suggested by the second interim report’s findings. The 
chapter also includes a discussion of the ongoing evaluation. 

Appendix A provides a detailed discussion of survey administration processes as well as response 
rates for the evaluation’s online surveys of principals on R-Tech campuses and program 
facilitators. The appendix also includes supplemental tables that provide additional information 
about results cited in report chapters and copies of the fall 2008 and spring 2009 surveys. 
Appendix B includes a discussion of survey administration procedures and response rates for the 
evaluation’s online survey of teachers working on R-Tech campuses. The chapter includes a copy 
of the teacher survey and supplemental tables that provide additional information about findings 
included in report chapters. 

Appendix C presents survey administration procedures and response rates for the spring 2009 
online survey of students who received R-Tech services during the program’s first 
implementation year, as well as a copy of the survey. 

Appendix D describes how site visit districts were identified and provides information about site 
visit activities, as well as a brief description of the programs implemented in site visit districts. 
Appendix E presents evaluation findings for technology-based dual credit programs offered as 
part of R-Tech, including the characteristics of students participating in dual credit courses, the 
challenges districts experience in implementing such courses, how challenges may be overcome, 
and students’ perceptions of dual credit offerings. 

Appendix F provides technical information about the HLM analyses presented in chapter 6, 
including supplementary tables that provide detailed information about regression variables and 
model results. 



Chapter 2 

The Characteristics of R-Tech Districts and Campuses 


In considering how R-Tech is implemented across districts and schools (Research Question 1), it is 
important to understand the characteristics of districts and schools that participate in the grant, as well as 
the characteristics of the students who attend them. In the competitive selection process for R-Tech 
grantees, TEA established that priority would be given to districts that (1) represented diverse 
geographical regions of the state, (2) demonstrated academic need in terms of state accountability ratings, 
(3) had sufficient infrastructure to implement R-Tech, and (4) had large proportions of middle school and 
high school students who would participate in the project (TEA, 2008a). Sixty-seven districts applied for 
R-Tech Cycle 1 grant funding, and TEA awarded grants to 64 districts in the spring of 2008. Of the 64 
original R-Tech grantees, one district opted not to participate in the grant and a second indicated it had 
deferred implementation to the 2009-10 school year - , which reduced the total number of Cycle 1 grantees 
to 62 districts and 115 campuses in 2008-09. 

This chapter presents information about the 62 districts awarded R-Tech funding, and the 115 schools 
participating in R-Tech. The chapter considers the geographical distribution of grantee districts, the 
accountability ratings of participating districts and schools, the characteristics of students and teachers in 
R-Tech schools, as well as the academic performance of R-Tech schools relative to TEA-identified peer 
comparison campuses 1 included in AEIS files and statewide averages. The chapter relies on data collected 
through Texas’ archival data sources PEIMS and AEIS for the 2007-08 school year - . 2 As noted in chapter 
1, PEIMS contains demographic and performance data on all students enrolled in Texas public schools, as 
well as information about schools’ organizational, staffing, and financial characteristics. AEIS contains 
information about the academic performance and accountability ratings of each of the state’s districts and 
schools. 

CHARACTERISTICS OF R-TECH DISTRICTS 

Of the 115 schools participating in R-Tech Cycle 1 grants, 59 were high schools, 46 were middle schools, 
7 were elementary schools, 3 2 campuses served PK through 12 students, and 1 campus served Grades 6 
through 12. While all R-Tech districts enrolled fewer than 5,000 students during the 2007-08 school year, 
there was a substantial range in district size, with the smallest district enrolling 208 students and the 
largest enrolling 4,854 students. On average, R-Tech districts enrolled 1,614 students, and R-Tech 
campuses enrolled 408 students. Statewide these averages were 3,900 and 617, respectively. 4 


'TEA-identified peer comparison campuses serve student populations that are similar to those served by R-Tech 
campuses. 

2 The 2007-08 PEIMS and AEIS data were the most current data available at the time of second interim report's 
writing. 

3 All seven elementary schools included Grade 6. In addition, one elementary school also included Grades 7 and 8. 
4 State district average excludes R-Tech districts; state campus average excludes R-Tech campuses and campuses 
characterized as elementary programs in 2007-08 AEIS data fdes. 
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Statewide Distribution of R-Tech Districts 


Analysis of R-Tech districts by Education Service Center (ESC) region is a useful means by which to 
examine the distribution of the program within the state. The Texas Education Code (TEC § 8.001) 
provides for the establishment of 20 regional ESCs throughout the state to assist districts with educational 
and operational matters. ESC’s regional boundaries are set by the Commissioner of Education and are 
designed such that each public school district has the opportunity to access ESC services. Figure 2.1 maps 
the regions served by each of Texas’ 20 ESCs. 



Figure 2.1. Texas’ Education Service Center regions 

Source: Texas Education Agency, 2008 
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As presented in Table 2.1, R-Tech districts were widely distributed across the state, with the largest 
proportion (9 districts or 15%) located in the area served by the ESC Region 10 (Richardson). The only 
regions of the state that did not include R-Tech districts were ESC Region 9 (Wichita Falls) and ESC 
Region 19 (El Paso). 


Table 2.1 

R-Tech Districts, by ESC Region: Fall 2008 and Spring 2009 


ESC Region 

Location 

Number of R-Tech 
Districts 

Percentage of 
R-Tech Districts 

Region 1 

Edinburg 

2 

3.2% 

Region 2 

Corpus Christi 

3 

4.8% 

Region 3 

Victoria 

1 

1.6% 

Region 4 

Houston 

5 

8.1% 

Region 5 

Beaumont 

5 

8.1% 

Region 6 

Huntsville 

4 

6.5% 

Region 7 

Kilgore 

6 

9.7% 

Region 8 

Mt. Pleasant 

3 

4.8% 

Region 9 

Wichita Falls 

0 

0.0% 

Region 10 

Richardson 

9 

14.5% 

Region 1 1 

Ft. Worth 

2 

3.2% 

Region 12 

Waco 

2 

3.2% 

Region 13 

Austin 

4 

6.5% 

Region 14 

Abilene 

2 

3.2% 

Region 15 

San Angelo 

2 

3.2% 

Region 16 

Amarillo 

4 

6.5% 

Region 17 

Lubbock 

1 

1.6% 

Region 18 

Midland 

2 

3.2% 

Region 19 

El Paso 

0 

0.0% 

Region 20 

San Antonio 

5 

8.1% 

Total 


62 

100.0% 


Source: 2007-08 Academic Excellence Indicator System Campus Reference file. 
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2008 Accountability Ratings of R-Tech Districts 

In addition to geographic diversity, R-Tech districts were selected because their 2007 accountability 
ratings indicated a need for educational services designed to improve student achievement. Table 2.2 
presents district level accountability ratings for R-Tech districts in 2008 as well as state averages. R-Tech 
districts lagged state averages in the proportions of districts rated Exemplary, Recognized, Academically 
Unacceptable, and in the proportion that were not rated. However, R-Tech districts exceeded the state 
average in terms of the proportion of districts rated Academically Acceptable. While about 6% of Texas 
districts received accountability ratings under the state’s alternative education accountability (AEA) 
procedures designed for districts that serve large proportions of students at risk of failure, no R-Tech 
district was characterized as an AEA district in 2008. 


Table 2.2 

District Accountability Ratings, by R-Tech District and State Averages: Fall 2007 and 
Spring 2008 



R-Tech Districts 

State 

Rating Category 

N 

% 

Average 1 

Standard Accountability Procedures 

Exemplary 

0 

0.0% 

3.7% 

Recognized 

13 

21.0% 

27.1% 

Academically Acceptable 

49 

79.0% 

60.3% 

Academically Unacceptable 

0 

0.0% 

2.6% 

Not Rated: Other 

0 

0.0% 

0.3% 

Alternative Education Accountability (AEA) Procedures 

AEA: Academically Acceptable 

0 

0.0% 

5.6% 

AEA: Academically Unacceptable 

0 

0.0% 

0.2% 

AEA: Not Rated-Other 

0 

0.0% 

0.3% 

Total 

62 

100.0% 

100.0% 


Source: 2007-08 Academic Excellence Indicator System District Reference file. 
“State averages omit R-Tech districts. 
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CHARACTERISTICS OF R-TECH CAMPUSES 


2008 Accountability Ratings of R-Tech Campuses 

Table 2.3 presents the 2008 campus-level accountability ratings for R-Tech schools as well as schools 
statewide. R-Tech campuses lagged the state average in the proportion of schools rated Exemplary, but 
exceeded state averages in terms of the proportion of schools rated Recognized and the proportion rated 
Academically Acceptable. In contrast to district-level ratings, a larger percentage of R-Tech campuses 
than campuses statewide received Academically Unacceptable ratings. While no R-Tech district was 
characterized as an AEA program in 2008, two R-Tech campuses were designed to serve at-risk students 
and registered as AEA programs — both of which received the AEA Academically Acceptable rating. 


Table 2.3 

Campus Accountability Ratings, by R-Tech Campus and State Averages: Fall 2007 and 
Spring 2008 



R-Tech Campuses 3 


Rating Category 

N 

% 

State Average 15 

Standard Accountability Procedures 

Exemplary 

3 

2.6% 

4.1% 

Recognized 

29 

25.4% 

17.4% 

Academically Acceptable 

70 

61.4% 

52.1% 

Academically Unacceptable 

10 

8.8% 

3.1% 

Not Rated: Other 

0 

0.0% 

12.4% 

Alternative Education Accountability (AEA) Procedures 

AEA: Academically Acceptable 

2 

1.8% 

10.2% 

AEA: Academically Unacceptable 

0 

0.0% 

0.4% 

AEA: Not Rated: Other 

0 

0.0% 

0.3% 

Total 

114 

100.0% 

100.0% 


Source: 2007-08 Academic Excellence Indicator System (AEIS) Campus Reference file. 

“There were no 2007-08 AEIS data for Mabank Intermediate School. 

b State averages omit R-Tech campuses and exclude campuses characterized as elementary programs in 
AEIS data files. 
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Student and Teacher Characteristics in R-Tech Districts 

Student characteristics. National statistics indicate that students attending rural schools are more likely 
to be White and less likely to be characterized as limited English proficient (LEP) than students in other 
locales. Nationally, rural schools enroll roughly similar proportions of special education students as other 
areas, and with the exception of suburban schools, rural schools enroll smaller proportions of 
economically disadvantaged students than other areas (NCES, 2007). As presented in Table 2.4, R-Tech 
campuses reflect national trends in terms of the types of students they enroll. Relative to state averages, 
R-Tech campuses enrolled a notably larger percentage of White students and a notably smaller percentage 
of Hispanic students, a smaller proportion of LEP students, a somewhat larger proportion of special 
education students, and a somewhat smaller percentage of economically disadvantaged students. 


Table 2.4 

R-Tech Student Characteristics, by R-Tech Campus and Statewide 
Averages: 2 Fall 2007 and Spring 2008 


Student Group 

R-Tech 

Campuses 2 

Statewide 

Average b 

African American 

9.3% 

14.8% 

Hispanic 

27.0% 

44.0% 

White 

62.6% 

37.3% 

Other 

1.0% 

3.8% 

Economic disadvantage 

46.9% 

49.0% 

Special education 

13.2% 

11.4% 

Limited English proficient 

3.2% 

8.2% 


Source: The Academic Excellence Indicator System (AEIS) 2008 Campus Student 
Statistics data file. 

“There are no 2007-08 AEIS data for Mabank Intermediate School. 

b State averages omit R-Tech campuses and exclude campuses characterized as 

elementary programs in AEIS data fdes. 
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Teacher characteristics. National statistics indicate that rural districts tend to employ fewer teachers 
from minority backgrounds, as well as more experienced teachers than districts in other locales (NCES, 
2007). Table 2.5 presents the characteristics of teachers working on R-Tech campuses with state averages. 
Similar to rural schools nationally, teachers in R-Tech districts were less likely to be from an ethnic 
minority and tended to have more experience, on average, than teachers statewide. Relative to state 
averages, R-Tech districts employed a larger percentage of teachers with 1 1 or more years of experience 
and a smaller percentage of teachers with 5 or fewer years of experience. A somewhat smaller percentage 
of teachers in R-Tech districts had advanced degrees, and, on average, R-Tech teachers worked with 
slightly smaller class sizes than teachers statewide. R-Tech districts also enjoyed lower rates of teacher 
turnover than did districts statewide. 


Table 2.5 

Teacher Characteristics, by R-Tech Campus and Statewide Average: Fall 2007 
and Spring 2008 


Teacher Characteristic 2 

R-Tech 

Campuses 

Average 15 

Statewide 

Average 0 

African American 

2.9% 

10.8% 

Hispanic 

9.7% 

17.4% 

White 

86.8% 

70.3% 

Teacher average years of experience d 

12.8 

11.5 

Teacher tenure in years' 1 

7.1 

6.8 

Beginning teachers 0 

7.9% 

8.6% 

1-5 years experience 

23.4% 

29.0% 

6-10 years experience 

18.2% 

19.2% 

1 1-20 years experience 

26.7% 

23.5% 

More than 20 years experience 

23.8% 

19.7% 

Teachers with no degree 1 

0.8% 

0.8% 

Teachers with advanced degrees 1 

17.4% 

21.6% 

Average beginning teacher salary d 

$33,590 

$37,666 

Average teacher salary' 1 

$42,280 

$44,646 

Teacher annual turnover rate 1 

19.1% 

20.9% 

Students per teacher (average) d 

12.0 

12.7 

Sources: 2008 Academic Excellence Indicator . 

System (AEIS) Campus Staff Statistics and 


District Staff Statistics data files. 

“The percentages of teachers by ethnicity, level of experience, and degree status are based on 
actual counts of teachers. The average years of experience, tenure in years, average salaries, 
annual turnover rate, and students per teacher are averages across campuses or districts. 
b There are no 2007-08 AEIS staff data for Mabank Intermediate School. 
c State averages calculated at the district level omit R-Tech districts, and averages calculated 
at the campus level omit R-Tech campuses, campuses characterized as elementary programs 
in AEIS data files, and masked data values. 
d 2007 TEA AEIS campus staff statistics fde. 

e A beginning teacher is a teacher with 0 years of reported experience. 

*2007 TEA AEIS district staff statistics file. 
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The Academic Performance of R-Tech Campuses 

The following sections provide baseline data 5 on R-Tech campuses’ academic performance. TAKS results 
are for the 2007-08 school year. Results for the other performance indicators are for the 2006-07 school 
year. Performance indicators include TAKS passing rates and commended performance rates, attendance, 
dropout, and mobility rates, advanced course completion rates, graduation rates, Recommended High 
School Program (RHSP) 6 completion rates, college entrance examination results, and college readiness 
indicators. Reported values are averages of campus-level measures for R-Tech campuses, the state (state 
average omitting R-Tech campuses and campuses characterized as elementary programs in AEIS data 
files), and TEA-identified peer campuses. 

TAKS performance. Table 2.6 and Figure 2.2 provide 2007-08 campus-level TAKS performance 
comparisons for students enrolled in R-Tech campuses. In the tested subject areas, overall TAKS 
performance in R-Tech schools was slightly above state averages in science, all tests taken, 
reading/English language aits (ELA), and social studies, and equal to the state average in mathematics 
and writing. Compared to peer campuses, R-Tech overall TAKS performance was somewhat lower in all 
tested areas. 

R-Tech 2007-08 commended performance rates tended to be lower than state and peer campus averages. 
Compared to state averages, R-Tech commended performance rates were from 2 to 4 percentage points 
lower. Compared to peer campus averages, R-Tech commended performance rates were from 2 to 5 
percentage points lower. Finally, the percentages of African American, Hispanic, White, and 
economically disadvantaged students passing all TAKS tests at R-Tech campuses were from 1 to 5 
percentage points lower than state averages. 


5 Baseline indicators are measures of school characteristics and performance prior to program implementation. Such 
measures provide a “baseline” from which to assess program effects. 

6 See http://www.tea.state.tx.us/teks/handbook/gradreq.pdf for the Recommended High School Program graduation 
requirements. 
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Table 2.6 

Average TAKS Performance, by R-Tech Campuses, Peer Comparison Campuses, and 
State Public School Campuses: Fall 2007and Spring 2008 


Category 

R-Tech 

Campus 

Average 

Peer Campus 
Average 

State 

Average 

R-Tech - 
Peer 

Difference 3 

R-Tech - 
State 

Difference 3 

Students Passing TAK! 

3 

All Tests Taken 

67% 

71% 

66% 

-4 

+ 1 

Reading/ELA 

91% 

93% 

90% 

-2 

+ 1 

Mathematics 

75% 

80% 

75% 

-5 

0 

Science 

71% 

75% 

69% 

-4 

+2 

Social Studies 

90% 

93% 

89% 

-3 

+ 1 

Writing 

92% 

95% 

92% 

-3 

0 

Students Attaining Commended Performance 

All Tests Taken 

10% 

12% 

13% 

-2 

-3 

Reading/ELA 

31% 

34% 

33% 

-3 

-2 

Mathematics 

19% 

24% 

23% 

-5 

-4 

Science 

16% 

18% 

18% 

-2 

-2 

Social Studies 

29% 

34% 

33% 

-5 

-4 

Writing 

31% 

36% 

34% 

-5 

-3 

Students Passing All Tests Taken 

African American 

51% 

NA 

56% 

NA 

-5 

Hispanic 

59% 

NA 

61% 

NA 

-2 

White 

74% 

NA 

77% 

NA 

-3 

Econ. disadvantage 

58% 

NA 

59% 

NA 

-1 


Sources: Data are from 2007-08 Academic Excellence Indicator System (AEIS) Campus Texas 
Assessment of Knowledge and Skills (TAKS ) data files. 

Notes. Data are averages across campuses. State averages omit R-Tech campuses characterized as 
elementary programs in AEIS data files. 

NA=Not applicable. Data are not available from the AEIS campus TAKS files. 

“The differences between R-Tech and peer campuses and R-Tech campuses and state averages are in 
percentage points. 
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Figure 2.2. Average campus-level TAKS passing rates, by R-Tech campuses, peer comparison 
campuses, and state averages: fall 2007 and spring 2008 

Sources: Data are from 2007-08 Academic Excellence Indicator System (AEIS) Campus Texas Assessment of 
Knowledge and Skills (TAKS) data files. 

Notes. Data are averages across campuses. State averages omit R-Tech campuses characterized as elementary 
programs in AEIS data files. 

Comparisons by grade provide a more detailed picture of TAKS performance. In Table 2.7, the 2007-08 
TAKS passing rates for R-Tech campuses were compared by content area, grade level, and comparison 
group. Grade-level comparisons for R-Tech campuses and state averages show that high school R-Tech 
campuses (Grades 9-11) performed from 0 to 5 percentage points higher than state averages. However, 
middle school campuses (Grades 6-8) performed from 1 percentage point lower to 1 percentage point 
higher than state averages. Grade-level comparisons for R-Tech campuses and peer campuses show that 
R-Tech campuses were performing below their peers at all grade levels and in all tested areas. The largest 
deficits were in mathematics and “all tests taken.” 
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Table 2.7 

Average TAKS Passing, by R-Tech Campuses, Peer Comparison Campuses, and State 
Public School Campuses and by Content Area and Grade Level: Fall 2007 and Spring 2008 


Grade 

R-Tech Campus 
Average 

Peer Campus 
Average 

State 

Average 

R-Tech - 
Peer 

Difference* 1 

R-Tech - 
State 

Difference 3 

Reading/ELA 

6 

92 

95 

92 

-3 

0 

7 

87 

91 

87 

-4 

0 

8 

94 

96 

93 

-2 

+ 1 

9 

90 

92 

85 

-2 

+5 

10 

90 

92 

86 

-2 

+4 

11 

90 

93 

88 

-3 

+2 

Mathematics 

6 

80 

86 

80 

-6 

0 

7 

79 

85 

79 

-6 

0 

8 

77 

84 

78 

-7 

-1 

9 

63 

69 

63 

-6 

0 

10 

64 

68 

63 

-4 

+1 

11 

79 

82 

75 

-3 

+4 

Science 

8 

67 

72 

67 

-5 

0 

10 

65 

67 

62 

-2 

+3 

11 

82 

83 

77 

-1 

+5 

Social Studies 

8 

88 

93 

88 

-5 

0 

10 

88 

90 

85 

-2 

+3 

11 

95 

97 

92 

-2 

+3 

Writing 

7 

93 

95 

92 

-2 

+ 1 

All Tests Taken 

6 

77 

84 

78 

-7 

-1 

7 

73 

79 

74 

-6 

-1 

8 

60 

67 

61 

-7 

-1 

9 

63 

68 

62 

-5 

+ 1 

10 

52 

55 

51 

-3 

+ 1 

11 

72 

74 

68 

-2 

+4 


Sources: Data are from 2007-08 Academic Excellence Indicator System (AEIS) Campus Texas Assessment 
of Knowledge and Skills data files. 

Notes. Data are averages across campuses. State averages omit R-Tech campuses and exclude campuses 
characterized as elementary programs in AEIS data files. 

“The R-Tech minus peer differences and the R-Tech minus state differences are in percentage points. 
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Attendance rates. Student attendance rates at R-Tech campuses were above the state average and slightly 
below the peer campus average (see Table 2.8). R-Tech campuses exceeded the state average by one and 
four-tenths percentage points and trailed the peer campus average by one half of a percentage point. 


Table 2.8 

Attendance Rates, by Comparison Group: Fall 2006 and 
Spring 2007 


Group 

Attendance Rate 

R-Tech campuses 

95.1% 

Peer comparison campuses 

95.6% 

State average 

93.7% 


Source: Data are from the 2007-08 Academic Excellence Indicator 
System (AEIS) Campus Non-Texas Assessment of Knowledge and 
Skills Performance Indicators data file and are for school year 2006-07. 

Notes. Data are averages across campuses. State averages omit 
R-Tech campuses and exclude campuses characterized as elementary 
programs in AEIS data fdes. 

Dropout rates. The 2006-07 R-Tech campus dropout rates at Grades 7 and 8 and Grades 9 through 12 7 
were lower than state averages but slightly higher than peer campus averages (See Table 2.9). The 
average R-Tech campus dropout rate for Grades 7 and 8 was lower than the state average by four-tenths 
of a percentage point, but higher than the peer campus rate by one-tenth of a percentage point. The Grades 
9 through 12 dropout rate for R-Tech campuses was lower than the state average by one and seven-tenths 
percentage points but exceeded the peer campus average by four-tenths of a percentage point. 


Table 2.9 

Dropout Rates, by Comparison Group: Fall 2006 and Spring 2007 


Group 

Dropout Rate 
Grades 
7 and 8 

Dropout Rate 
Grades 
9 Through 12 

R-Tech campuses 

0.1% 

2.1% 

Peer comparison campuses 

0.0% 

1.7% 

State average 

0.5% 

3.8% 


Source: Data are from the 2007-08 Academic Excellence Indicator System (AEIS) 

Campus Non-Texas Assessment of Knowledge and Skills Performance Indicators data 
file and are for school year 2006-07. 

Notes. Data are averages across campuses. State averages omit R-Tech campuses and 
exclude campuses characterized as elementary programs in AEIS data files. 

Student mobility. A student is considered to be mobile if he or she has missed 6 or more weeks at a 
particular' school. Campus student mobility is determined by dividing the number of mobile students 
during the school year by the number of students who were in membership at any time during the school 
year (2006-07 AEIS Glossary, TEA). Figure 2.3 shows the average campus mobility rates of R-Tech 
schools, peer campuses, and the state average (with R-Tech and elementary campuses removed) from 
2004 through 2007. Students were less mobile at R-Tech campuses compared with state averages, but 
slightly more mobile than students at peer campuses. The mobility rates at R-Tech campuses were about 
15 to 16 percentage points lower than the state average, but about 1 to 2 percentage points higher than the 
mobility rates at peer campuses. 


7 TEA reports separate dropout rates for Grades 7 and 8 and for Grades 9 throughl2. Reports for secondary campuses 
evaluated under standard accountability procedures show both of these rates. 
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Figure 2.3. Campus-level mobility rates by R-Tech campuses, peer campuses, and state averages: 
fall 2007 and spring 2008. 

Source: Data are from 2007-08 Academic Excellence Indicator System (AEIS) Campus Student Statistics data file. 
Notes. Data are averages across campuses. State averages omit R-Tech campuses and exclude campuses 
characterized as elementary programs in AEIS data files. 

Advanced course performance. Table 2.10 presents information on the percentage of students who 
completed and received credit for at least one advanced course at R-Tech campuses that enrolled students 
in Grades 9 or higher. Advanced courses include Advanced Placement (AP) and International 
Baccalaureate courses along with higher-level core content area courses (e.g., pre-calculus, 
research/technical writing, economics advanced studies), advanced elective courses (e.g., French IV, 
Theatre Arts IV, Music IV Jazz Band), and dual credit courses for which a student may receive both high 
school and college credit. Advanced course completion is calculated by dividing the number of students 
who received credit for at least one advanced or dual credit academic course by the number of students 
who received credit for at least one course during the school year. 


Table 2.10 

Advanced Course Completion Rates, by Comparison Group: Fall 2006 and Spring 2007 


Group 

R-Tech 

Campuses 

Peer Comparison 
Campuses 

State 

Average 

African American 

11.3% 

NA 

12.1% 

Flispanic 

12.7% 

NA 

13.2% 

White 

21.3% 

NA 

19.5% 

Economically disadvantaged 

11.8% 

NA 

11.9% 

All students 

17.7% 

18.3% 

15.8% 

Source: Data are from the 2007-08 Academic Excellence Indicator System (AEIS) Campus 

Son-Texas 


Assessment of Knowledge and Skills (TAKS) Performance Indicators data file and are for school year 2006-07. 
Notes. Data are averages across campuses. State averages omit R-Tech campuses and exclude campuses 
characterized as elementary programs in AEIS data files. 

NA=Not applicable. Data are not available from the AEIS campus non-TAKS performance indicators data file. 
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Compared to peer campuses, R-Tech campuses had a slightly lower average of advanced course 
completions (less than 1 percentage point lower). However, compared to the state average, R-Tech 
campuses had a slightly higher percentage of advanced course completions (nearly 2 percentage points 
higher). R-Tech campuses also had a slightly higher percentage of advanced course completions for 
White students. However, Hispanic, economically disadvantaged, and African American students at 
R-Tech campuses completed advanced courses at slightly lower rates than the state average. 

Graduation and Recommended High School Program completion rates. Outcome measures such as 
graduation rates and RHSP 8 completion rates also reflect student and campus performance. Findings for 
these measures by comparison group are presented in Table 2.1 1. The 2006-07 R-Tech high school 
graduation rate was higher than the state overall, and similar - to the peer campus rate. Specifically, the 
R-Tech graduation rate was 88%, while the state rate was 75%, and the peer campus rate was 88%. 
Another measure of academic readiness is the RHSP completion rate. In 2006-07, the RHSP required 24 
credits and more rigorous elective courses (e.g., fine arts, languages other than English) than the 22-credit 
minimum graduation plan. 9 Compared to the state average, a higher percentage of R-Tech students 
completed the RHSP in 2006-07. Compared to peer campuses, a lower percentage of R-Tech students 
completed the RHSP in 2006-07. 


Table 2.11 

Graduation Rates and Recommended High School Program Completion Rates, 
by Comparison Group: Fall 2006 and Spring 2007 


Group 

Graduation 

Rate 

RHSP Completion 
Rate 

R-Tech campuses 

88.2% 

73.3% 

Peer comparison campuses 

88.1% 

77.1% 

State average 

75.3% 

69.0% 


Source: Data are from the 2007-08 Academic Excellence Indicator System (AEIS) Campus 
Non-Texas Assessment of Knowledge and Skills Performance Indicators data file and are for 
school year 2006-07. 

Notes. Data are averages across campuses. State averages omit R-Tech campuses and exclude 
campuses characterized as elementary programs in AEIS data files. 

The graduation rate at a campus is calculated by dividing the number of students who received 
a high school diploma by the end of the cohort’s graduation year by the number of students in 
the original cohort. The Recommended High School Program rate is calculated by dividing the 
number of graduates with graduation codes for Recommended High School Program or 
Distinguished Achievement Program by the number of graduates. 

College entrance examinations. College enhance examination scores are reported to TEA; the Agency 
then reports the percentage of students taking examinations and average examination scores by campus. 
Data are reported when students are scheduled to be seniors, regardless of when examinations are taken. 
One factor that may influence college entrance examination results is the percentage of students taking 
the examinations. Lower percentages of students taking examinations may be associated with higher 
average scores. As presented in Table 2.12, the percentage of R-Tech seniors taking college enhance 
examinations was 63% in 2006-07. This was higher than the state average (56%) but lower than the peer 
campus average (67%). SAT average scores were lowest for R-Tech campuses (953 vs. 973 for peer 
campuses and 962 for the state). ACT average scores were highest for peer campuses (20.0), followed by 
R-Tech campuses (19.6) and the state (19.6). 


8 See http://www.tea.state.tx.us/teks/handbook/gradreq.pdf for the Recommended High School Program graduation 
requirements. 

9 Texas expanded the RHSP to include 26 credits in 2007-08. 
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Table 2.12 

SAT and ACT College Entrance Examination Scores, by Comparison Group: Fall 2006 
and Spring 2007 


Group 

Percentage 
Taking SAT or 
ACT 3 

SAT 

Average 

ACT 

Average 

R-Tech campuses 

62.9% 

953 

19.6 

Peer comparison campuses 

67.2% 

973 

20.0 

State average 

55.7% 

962 

19.6 


Source: Data are from the 2007-08 Academic Excellence Indicator System (AEIS) Campus College 
Admissions, College -ready Graduates data file and are for school year 2006-07. 

Notes. Data are campus averages across students who took the Scholastic Aptitude Reasoning Test (SAT) 
or the ACT. State averages omit R-Tech campuses and exclude campuses characterized as elementary 
programs in AEIS data files 

“The percentage is calculated by dividing the number of graduates who took either the SAT or ACT by 
the number of non-special education graduates. 

College readiness. The 2006-07 AEIS data included a new indicator of college readiness — the percentage 
of college-ready graduates. This indicator is a measure of progress toward preparation for postsecondary 
success. To be considered college ready as defined by this indicator, a graduate must have met or 
exceeded specified criteria on the exit-level TAKS test, SAT, or ACT. These criteria are listed in Table 
2.13. 


Table 2.13 

College-Readiness Indicators and Criteria, by Exam and Subject Area: Fall 2006 and Spring 2007 


Subject 

Exit-level TAKS 


SAT 


ACT 

ELA 

>= 2200 scale score on 
ELA test AND a “3” or 
higher on the essay 

OR 

>= 500 on Critical 
Reading AND >= 1070 
Total 

OR 

>= 19 on English AND 
>= 23 Composite 

Mathematics 

>= 2200 scale score on 
mathematics test 

OR 

>= 500 on Math AND 
>= 1070 Total 

OR 

>= 19 on Math AND 
>= 23 Composite 


Source: AEIS Glossary, p.10, November 2007. 
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As Table 2.14 indicates, the percentages of 2006-07 R-Tech graduates who were college ready were 
similar to state averages but lower than peer campus averages. For example, in mathematics, 49% of 
R-Tech graduates were college ready compared to 49% across the state and 54% at peer campuses. In 
reading, 42% of R-Tech graduates were college ready compared to 41% across the state and 46% at peer 
campuses. 

Table 2.14 

College Readiness Indicators, by Comparison Group: Fall 2006 and Spring 2007 


Group 

College Ready 
Mathematics 

College Ready 
Reading 

College Ready 
Both Subjects 

R-Tech campuses 

48.9% 

41.6% 

28.9% 

Peer comparison campuses 

54.3% 

45.8% 

33.0% 

State average 

48.6% 

41.4% 

29.4% 


Source: Data are from the 2007-08 Academic Excellence Indicator System ( AEIS) Campus College 
Admissions, College-Ready Graduates data file and are for school year 2006-07. 

Notes. Data are averages across campuses. State averages omit R-Tech campuses and exclude campuses 
characterized as elementary programs in AEIS data files. 

SUMMARY 

Compared to state averages, R-Tech campuses enrolled a higher percentage of White students but lower 
percentages of African American, Hispanic, and LEP students. R-Tech campuses and state averages were 
similar for percentages of economically disadvantaged and special education students. Compared to state 
averages, R-Tech campuses employed higher percentages of White teachers and lower percentages of 
minority teachers. R-Tech teachers tended to have more experience, but proportionately fewer had 
advanced degrees. R-Tech teacher salaries and turnover rates and class sizes were lower than state 
averages. 

R-Tech campuses performed better than state averages on most of the indicators of academic 
performance. For example, R-Tech campuses had generally higher TAKS passing rates and a higher 
attendance rate, a lower dropout rate, a lower mobility rate, a higher advanced course completion rate, a 
higher graduation rate, a higher RHSP completion rate, and a higher percentage of seniors taking college 
entrance exams. However, R-Tech campuses did not perform as well as peer campuses on any of these 
indicators. R-Tech campuses had TAKS passing rates that were below peer averages in all tested areas, 
lower SAT and ACT average scores, and lower percentages of college-ready graduates. It is important to 
note that the R-Tech selection process gave priority to campuses that demonstrated “an overall academic 
need.” With this as a mitigating factor, it is understandable why R-Tech campuses did not perform as well 
as demographically similar campuses. 
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Chapter 3 

R-Tech Implementation 


Considerable research has established that the manner in which schools implement programs designed to 
improve student achievement is closely associated with observed outcomes, and that commitment to 
program goals at the district and campus level, as well as teacher buy-in and support are critical to 
implementation quality (Berman & McLaughlin, 1978; Bifulco, Duncombe, & Yinger, 2005; Borman, 
2005; Borman, Hewes, Overman, & Brown, 2003; Datnow, Borman, & Stringfield, 2000; Vernez, 

Karam, Mariano, & DeMartini, 2006; Yap, 1996). However, as discussed in chapter 1, supplemental 
instructional programs such as R-Tech are rarely implemented in a uniform manner. Differences in SES 
vendors, program goals, and implementation requirements make it difficult for researchers to identify 
whether supplemental programs are implemented effectively and whether implementation quality affects 
student outcomes. 

This chapter provides information about how districts implement their R-Tech programs (Research 
Question 1), and considers the following questions: 

• What types of programs do R-Tech grantees implement? 

• What banders limit the implementation of R-Tech programs in grantee schools? 

• How are banders to implementation overcome? 

In addressing R-Tech implementation strategies, the chapter provides an overview of the implementation 
requirements included in R-Tech’s enabling legislation and in TEA grant requirements. It defines the five 
types of programs identified by the evaluation and describes how districts structured their approaches to 
implementation, including stakeholder roles and responsibilities. The chapter further considers the 
banders districts encounter in implementing R-Tech and how barriers may be overcome. 

DATA SOURCES 

To answer research questions, the chapter combines data collected through analysis of R-Tech 
documents, including grant applications and progress reports; surveys of principals, R-Tech facilitators, 
and teachers; as well as qualitative data collected through interviews with principals and program 
facilitators and focus group discussions with teachers conducted as part of site visits to eight R-Tech 
districts in spring 2009. The chapter also incorporates data provided by districts through TEA’s R-Tech 
Student Upload data system. 
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OVERVIEW OF R-TECH PROGRAM REQUIREMENTS 


In enacting R-Tech, legislators required that grantee districts provide technology-based supplemental 
instruction to students in Grades 6 through 12, and provide access to such instruction for at least 10 hours 
each week (TEC §29. 919). 1 As noted in chapter 1, districts may use R-Tech funding to provide: 

• Research based instructional support, 

• Teacher training, 

• Academic tutoring or counseling, 

• Distance learning opportunities in the core content areas or in foreign languages, and 

• Dual credit coursework in the core content areas or in foreign languages. 

TEA grant application requirements further specify that R-Tech resources may “only” be used to support 
ELA, social studies, mathematics, science, or languages other than English (TEA, 2008b, p. 6). 
Application guidelines state that grantee districts provide students participating in R-Tech with a Personal 
Education Plan, or PEP, used to track students’ progress, and require that districts employ program 
facilitators to support program implementation. R-Tech facilitators are expected to receive training in 
supplemental instruction products and support program implementation by monitoring student progress, 
managing reporting requirements, and ensuring student access to R-Tech resources (TEA, 2008b). 

THE TYPES OF PROGRAMS IMPLEMENTED THROUGH R-TECH 

As discussed in chapter 1, researchers analyzed R-Tech grant applications and progress reports submitted 
to TEA in order to classify R-Tech programs in terms of the types of instruction provided. This analysis 
produced the following categories of R-Tech programs: 

1. Self-paced instruction focused on tutoring, remediation, or credit-recovery; 

2. Dual credit and distance learning programs; 

3. One-to-one tutoring with online instructional support; 

4. School-wide technology immersion programs; and 

5. Programs that provide students with iPods loaded with instructional content. 

The identified categories are not discrete across districts. That is, districts may implement more than one 
type of program. R-Tech allows districts to implement different types of programs at the middle school 
and high school level, depending on student needs. For example, a district may implement a dual credit 
program for high school students and a self-paced tutoring program for middle school students. 

Supplemental vs. Non-Supplemental Instruction 

Although R-Tech was intended to provide supplemental instruction offered outside of regularly scheduled 
classes, the analysis of district grant applications and progress reports revealed that many districts chose 
to include R-Tech as part of regular instruction (i.e., non- supplemental instruction). For example, some 
districts used R-Tech funding to upgrade and expand computer labs, in which teachers scheduled class 
time to use R-Tech resources to support course content or to provide tutoring in preparation for TAKS 
testing. Some districts used R-Tech funding to purchase laptops that students used in class on a regular 
basis. Across the 63 Cycle 1 R-Tech districts, 34 districts (54%) implemented supplemental programs, 
and 25 districts (40%) implemented R-Tech as part of the regularly scheduled school day during the 
program’s first year. Four districts (6%) had not fully implemented R-Tech in spring 2009, and it was 
unclear whether they would offer supplemental or non-supplemental programs. 


'information on the number of hours students access R-Tech each week is included in chapter 4 (see Table 4.2) 
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The sections that follow provide more information about the types of programs districts implemented 
during R-Tech’s first year, and whether R-Tech services were implemented as supplemental or non- 
supplemental programs. The categories of programs identified in this chapter are also incorporated in the 
preliminary analyses of R-Tech’s effects on student TAKS outcomes (chapter 6) and the program’s cost 
effectiveness (chapter 7). 

Categories of R-Tech Programs 

Table 3.1 provides information about the percentage of districts implementing each type of R-Tech 
program, and indicates that most districts designed self-paced programs using R-Tech funding (87%). A 
smaller percentage of districts implemented dual credit or distance learning programs (30%), and two 
districts (3%) implemented each of the remaining program types. The following sections describe the 
characteristics of R-Tech programs included in each category. 


Table 3.1 

The Percentage of R-Tech Districts by Program Type: Fall 2008 and 
Spring 2009 


Types of R-Tech Programs 

Districts 

(N=63) 

Self-paced instruction 

87.3% 

Dual credit and/or distance learning 

30.1% 

One-to-one tutoring with online instructional support 

3.2% 

Technology immersion 

3.2% 

iPods with instructional content 

3.2% 


Sources: District grant applications and progress reports. 

Note. Percentages will not total to 100; districts may implement more than one type 
of program. 


Self-paced instructional programs. Self-paced instructional programs provide students with online 
lessons and tutorials that students work through at their own pace. Many programs provide diagnostic 
assessments in course content and route students to specific lessons that address areas of weakness. When 
students demonstrate competency in course content, generally through a test scored by the program, they 
move to subsequent lessons that address more advanced skills. 

A range of R-Tech programs are included in the self-paced category in large paid because the software 
packages that provide technology-based self-paced instruction are diverse in terms of the types of 
instruction offered and the subject areas addressed. For example, some self-paced programs provide 
technology-based tutoring, remediation, and credit-recovery programs for many subject areas (e.g., 
PLATO, OdysseyWare). Other programs focus on TAKS remediation across subject areas (e.g., 
A+nywhere Learning System), while others focus on specific subjects, such as ELA (e.g., MyStudy Flail) 
or math (e.g., iSuceedMath). Still other programs provide self-paced instruction in social and behavioral 
issues (e.g., RippleEffects). Of the 55 districts that implemented self-paced instruction as paid of R-Tech, 
most implemented supplemental programs (60% or 33 districts), providing instruction before or after 
school or at a time when students were not attending regular classes (e.g., study hall). 

Dual credit and distance learning programs. Technology-based dual credit and distance learning 
programs enable high school students to take courses for which they earn both high school and college 
credit. Courses are generally taught by college faculty, and students participate in lessons online or 
through video conferencing arrangements. Such courses often require that students participate in online 
discussions through “chat rooms” and to submit coursework and complete exams electronically. Fligh 
school students who participate in dual credit courses are not required to pay college tuition, and districts 
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must cover the costs of college textbooks. Dual credit courses are increasingly seen as an effective means 
to increase the rigor of high school curricula and to enable high school students to accrue college credit. 
Such programs offset college costs by enabling students to earn credit without paying college tuition and 
fees, and may increase student interest in postsecondary education (Maloney, Lain, & Clark, 2009). 

The districts that implemented dual credit and distance learning programs during R-Tech’s first year 
partnered with a range of higher education institutions, including regional community colleges and state 
colleges and universities, as well as the Texas Rural Educational Development Consortium, or TxRED, to 
provide dual credit courses. Of the 18 districts offering dual credit instruction as part of R-Tech, 60% (12 
districts) offered supplemental dual credit courses that students took in addition to their regularly 
scheduled classes. A comprehensive discussion of the implementation of technology-based dual credit 
offerings, including the characteristics of students who participate in coursework, the challenges to 
implementation and students’ perceptions of courses is included in Appendix E. 

One-to-one tutoring with online instructional support. Two R-Tech districts plan to offer R-Tech as 
tutoring programs in which tutors provide students with one-to-one instruction complemented by 
technology-based instructional support. Both districts have contracted services from TxRED, although 
neither district implemented its program for students during R-Tech’s first year. At the time of this 
report’s writing, it was not known whether one-to-one tutoring and support would be provided as 
supplemental programs or whether such support would be implemented as paid of the regular school day; 
however, data collected for the evaluation’s final report (fall 2010), should clarify whether such programs 
are implemented as part of regular instruction or as supplemental programs. 

Technology immersion programs. Technology immersion programs generally provide all students and 
teachers with laptop computers loaded with instructional resources, as well as access to the Internet 
during the school day. Many such programs allow students to take computers home, extending access to 
instructional resources, and students with home Internet access may also use laptops for online 
instructional activities. Although challenging to implement effectively, research has shown that 
technology immersion programs have the potential to substantially improve students’ academic outcomes, 
engagement in schooling, and proficiency using technology resources (Shapley, Sheehan, Maloney, & 
Caranikas- Walker, 2008). Both R-Tech districts offering technology immersion programs implemented 
non- supplemental programs, providing students with access to laptops and technology-based instructional 
resources as paid of regularly scheduled classes. 

iPods loaded with instructional content. Two Cycle 1 districts used R-Tech resources to provide 
students with iPods loaded with instructional programs. One district targeted its program to ELLs in an 
effort to improve students’ language skills. The second district provided middle school students with 
iPods loaded with content in the core subject areas, as well as music and physical education content. 
Teachers provided students with assigned videos and followed up with activities to ensure students’ 
understood the content they viewed. Both programs were implemented outside of the regular school day, 
and students were expected to view lessons at home. An ancillary effect of the ELL program was that 
students’ parents also had access to lessons, which according to school administrators, has improved 
parents’ language skills. 

R-Tech Software Selections 

The variety of software packages districts purchased using R-Tech funds also reveals the range of 
programs implemented through the grant. This section examines the software packages that districts 
indicated they would purchase in grant applications relative to their actual purchases reported in progress 
reports submitted to TEA in spring 2009. In implementing R-Tech, districts were permitted to select up to 
two software vendors — one vendor for the middle school and a second for the high school. 
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R-Tech districts initially selected a total of 28 separate software vendors to provide supplemental 
instruction; however, analysis of progress reports submitted to TEA showed that vendor selection 
expanded to include more than 40 software programs over the course of the first implementation year. 
Information included in district progress reports revealed that some districts changed vendors because of 
dissatisfaction with services or incompatible software programs. Table 3.2 presents information about the 
software programs that were initially selected by at least three R-Tech districts. Findings indicate that a 
small proportion of districts changed vendor selections, but that most districts implemented programs as 
planned. 

The largest proportions of districts (15%) selected and implemented A+nywhere Learning System. 
A+nywhere Learning System is marketed as a TAKS- and Texas Essential Knowledge and Skills 
(TEKS)-aligned program that provides diagnostic assessments and self-paced tutorials across a broad 
range of subject areas. Five districts (8%) implemented PLATO Learning systems. PLATO Learning 
provides self-paced remediation and credit-recovery programming for students in Grades 6 through 12 
through online and distance learning formats. Four districts purchased Achieve TeenBiz3000, which 
provides self-based tutoring in ELA. Four districts selected Apple’s suite of programs, and four districts 
selected OdysseyWare, both of which provide self-paced programming across subject areas. 


Table 3.2 

Vendors Selected for R-Tech Implementation, as a Percentage of Districts by 
Districts’ Plans and Implemented Programs: Fall 2008 and Spring 2009 


Vendor 

Districts’ Plans 
(N=62) a 

Districts’ Programs 
(N=62) a 

A+nyWhere Learning System 

14.5% 

14.5% 

PLATO Learning 

11.3% 

8.1% 

Achieve TeenBiz 3000 

8.1% 

6.5% 

Apple 

4.8% 

6.5% 

Odyssey Ware 

4.8% 

6.5% 

Agile Mind 

4.8% 

4.8% 

Ascend 

4.8% 

4.8% 

Compass Learning Odyssey 

4.8% 

4.8% 

Epic Learning 

4.8% 

4.8% 

NovaNET 

6.5% 

4.8% 

Apangea 

4.8% 

3.2% 

New Century 

4.8% 

3.2% 

Successmaker 

4.8% 

1.6% 


Source: Analysis of R-Tech district applications, fall 2008; Analysis of R-Tech District Progress 
Reports, spring 2009. 

Note. Percentages will not total to 100. Districts may select up to two vendors and only the vendors 

which three or more districts planned to use or actually used are displayed. 

a At the time of progress reports, one district was not implementing a program and could not be 

analyzed. 

Site visit interviews with principals and R-Tech facilitators asked about the processes districts used to 
select technology vendors. Principal and facilitator comments indicated that districts prioritized the 
provision of strong technical support, as well as user-friendly content-oriented programs in selecting 
software. A facilitator in a district implementing a self-paced program explained that the selection of the 
R-Tech vendor was the shared responsibility of district administrators, technology coordinators, and 
teachers, noting “it was the technical support from the company that impressed us.” Facilitators in several 
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other districts commented that teachers tested products and selected programs that “cover[ed] different 
content areas” and were easy for students to use. 

Subject Areas Addressed by R-Tech 

Table 3.3 presents the percentage of districts providing R-Tech support across subject areas and the 
percentage of students receiving services by subject area during R-Tech’ s first implementation year (May 
2008-May 2009). Results indicate that nearly all districts provided support in math (95%) and most 
students used R-Tech services to support their understanding of math concepts (70%). Districts also 
emphasized support in core subject areas such as ELA (86%), science (78%), and social studies (76%). 
While 28% of districts offered R-Tech services in languages other than English, a notably small 
proportion of students (3%) received instruction in this area. 


Table 3.3 

Subject Areas Addressed by R-Tech Programs, as a Percentage of 
Districts and Students: Summer 2008, Fall 2008, and Spring 2009 


Subject Areas 

Districts' 1 

Students' 5 

Addressed by R-Tech 

(N=58) 

(N= 14,849) 

English Language Arts 

86.2% 

45.9% 

Mathematics 

94.8% 

69.6% 

Science 

77.6% 

31.2% 

Social Studies 

75.9% 

13.4% 

Language Other than English 

27.6% 

2.9% 

Other Subject Area 

1.7% 

0.1% 


Sources: Texas Education Agency R-Tech Student Upload data, summer 2008, 
fall 2008, and spring 2009. 

Notes. Percentages sum to more than 100 because districts may have offered 
support in more than one subject area and students may have received 
instruction in more than one subject area. 

“The number of districts providing upload data was 58. Overall, 62 districts 
participated in R-Tech during the program’s first implementation year. 
b Students included in the analysis received R-Tech services in at least one of 
the following periods: summer 2008, fall 2008, and spring 2009. Students 
receiving services across multiple periods are counted only once in the analysis. 

HOW DISTRICTS IMPLEMENT R-TECH 

Drawing on the surveys of R-Tech facilitators, and principals and teachers on R-Tech campuses, as well 
as information collected through site visit interviews and focus group discussions, this section considers 
how districts implemented R-Tech. The discussion examines how R-Tech facilitators were selected and 
Rained; the roles of principals, R-Tech facilitators, and teachers in putting programs in place; how PEPs 
were developed; and how R-Tech goals aligned with those of implementing districts. 

Selecting and Training R-Tech Facilitators 

In interviews, facilitators said they were selected for the position because they had experience working in 
computer labs, had technology certifications, or had volunteered for the job. Survey results for facilitators, 
however, suggest that few had technology certifications. Table 3.4 presents the percentage of R-Tech 
facilitators who indicated they had technology certifications for the fall 2008 and spring 2009 survey 
administrations. Across both surveys, about 84% of facilitators indicated they had no certification. 
Although the percentages of facilitators with technology application certifications (Grades 8-12 or all 
levels) increased somewhat from fall to spring, the percentages of facilitators with master technology 
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teacher or computer science certifications declined across administration periods. These shifts are likely a 
reflection of district turnover unrelated to R-Tech. 


Table 3.4 

R-Tech Facilitators’ Technology Certification, as a Percentage of 
Respondents: Fall 2008 and Spring 2009 


Type of Certification 

Fall 2008 
(N=71) 

Spring 2009 
(N=61) 

Not certified 

84.5% 

83.6% 

Technology applications. Grades 8-12 

8.5% 

11.5% 

Technology applications, all levels 

2.8% 

4.9% 

Master technology teacher 

2.8% 

0.0% 

Computer science, Grades 8-12 

1.4% 

0.0% 


Sources'. R-Tech Principal/Facilitator Survey: fall 2008 and spring 2009. 


The surveys also asked facilitators to indicate the amount of training they received to support R-Tech 
implementation. In response to the spring survey facilitators reported receiving an average of about 14 
hours of training across R-Tech’ s first implementation year. Sixty-one facilitators provided written 
responses to an open-ended survey item asking about the content of their training. Of these. 49% 
indicated that they participated in training provided by R-Tech vendors, and about 11% reported receiving 
training from district technology coordinators. Facilitators wrote that vendor-provided training was 
program specific and focused on the introduction of software packages, approaches to monitoring 
students’ progress, and solving implementation challenges. District training was generally provided by a 
technology coordinator and focused on the use of hardware and the development of technical support 
skills. 

Implementation Roles 

The spring 2009 survey of principals and R-Tech facilitators asked respondents to indicate the degree to 
which they were involved in implementing R-Tech. The survey provided a list of implementation tasks 
and asked respondents to rate their level of involvement using the following responses: no involvement , 
minor involvement, moderate involvement, or substantial involvement. Figure 3.1 presents the summed 
percentages of principals and facilitators who reported moderate or substantial involvement for each task. 
Summed percentages represent the percentage of respondents indicating moderate involvement plus the 
percentage of respondents indicating substantial involvement. (See supplemental Table A.4 in Appendix 
A for individual percentages by all response categories for each implementation task and respondent 
group.) 
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Planning Identifying students Monitoring Communicating Participating in Developing PEPs Providing technical 
implementation students' use with parents training support 

□ Principals (N=75) □ Facilitators (N=61 ) 


Figure 3.1. Percentage of principals and facilitators indicating moderate or substantial involvement 
in activities, as a percentage of respondents: spring 2009. 

Source: R-Tech Principal and Facilitator Survey, spring 2009. 

Note. Summed percentages represent total of two response categories: ( 1 ) the percentage of respondents who 
indicated moderate involvement, and (2) the percentage of respondents who indicated substantial involvement in 
activities. 

Results indicate that principals tended to have a greater role in planning R-Tech, identifying students, and 
communicating with parents about R-Tech services, but that facilitators had a greater role in day-to-day 
implementation tasks, such as monitoring student use of technology resources, participating in training, 
and providing technical support for the grant. 

Principals’ roles in implementing R-Tech. As indicated in Figure 3.1, principals were less involved in 
R-Tech implementation than R-Tech facilitators, and their greatest involvement was with tasks that were 
necessary to get R-Tech stalled, but were not ongoing responsibilities (e.g., planning and 
communication). Principals’ comments during site visit interviews support this understanding. When 
asked to describe his participation in the planning process, one principal said: 

I guess you could just call me a team member. ... I think if you had [to select a] lead person, it 
would be either [name omitted] or [name omitted], and the rest of us were just on the team as we 
put the grant together. 

Two principals said they assisted in identifying which student groups and subject areas to address through 
R-Tech, but they did not select individual student participants or R-Tech vendors. Several interviewed 
principals said they had “very little” involvement in R-Tech implementation and considered their role to 
be one of encouragement and support for those who actively implemented programs. “My only role is 
allowing [the program’s] use,” noted one principal. 
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Facilitators’ roles in implementing R-Tech. Not surprisingly, R-Tech facilitators shouldered more 
responsibility for the ongoing implementation of districts’ programs. During site visit interviews, 
facilitators provided greater detail about their roles. One facilitator noted: 

[My role is] to develop, write, and coordinate all of the grant activities.... I have done everything 
from ordering the materials, doing the budget, setting up summer school, monitoring the summer 
school (for free, I might add) .... [I work] after hours, weekends, and summers .... I work with the 
principals, teachers, and students on any of their needs that involve the program. 

Facilitators in districts implementing self-paced instructional programs were more likely to say they 
supervised student work. For example, one such facilitator said, “I walk around and monitor the students 
and help them if they need help.” In contrast, facilitators working in schools implementing school-wide 
technology immersion programs were more likely to provide technical support, manage access to 
resources, and support teachers’ classroom use of technology. Facilitators in districts using iPods said 
they researched and located lesson content and loaded it on iPods, met with students to assess learning, 
monitored student progress, and managed the care of equipment. 

Teachers’ roles in implementing R-Tech. The fall and spring surveys asked teachers to indicate their 
roles in R-Tech by selecting no role, a minor role, a moderate role, or a substantial role for a variety of 
activities related to planning and implementing the grant. Across survey administrations, more than 85% 
of respondents indicated that they had no role in planning R-Tech, and more than 50% indicated they had 
no role in implementing the grant (see supplemental Tables B.3 and B.4 in Appendix B). The percentages 
of teachers responding that they had no role in R-Tech decreased somewhat across the fall to spring 
survey administrations, which likely reflects an increased awareness of R-Tech in spring 2009. 

Table 3.5 presents the summed percentages of teachers who indicated they had a moderate or substantial 
role in R-Tech activities for both survey administrations. Summed percentages represent percentage of 
teachers indicating a moderate role plus the percentage of teachers indicating a substantial role in 
activities. (Supplemental Tables B.3 and B.4 in Appendix B presents individual percentages by all 
response categories for each implementation activity across both survey administrations.) Results indicate 
that few surveyed teachers were involved in planning R-Tech (fewer than 5% of respondents across 
planning activities and survey administrations). Teachers also had limited roles in implementing R-Tech. 
A third (33%) had supervised students participating in R-Tech services by spring 2009, and about 30% 
had provided tutoring. Notably, less than 20% of surveyed teachers reported participating in the 
remaining implementation activities during R-Tech’ s first year. 
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Table 3.5 

Summed Percentages of Teachers’ Reporting Moderate or Substantial Involvement in Planning 
and Implementing R-Tech, as a Percentage of Respondents: Fall 2008 and Spring 2009 


Teacher Roles 

Fall 2008 
(N=l,213) 

Spring 2009 
(N=568) 

Planning Roles 

Decision to apply for grant 

2.8% 

4.2% 

Selection of vendors 

1.8% 

3.7% 

Drafting the grant application 

0.9% 

2.6% 

Implementation Roles 

Supervise or monitor students 

21.6% 

33.4% 

Provide tutoring to students 

20.2% 

29.5% 

Monitor Personal Education Plans 

15.1% 

18.1% 

Communication with parents 

11.2% 

15.0% 

Develop Personal Education Plans 

10.6% 

13.5% 

Identification of students 

10.9% 

13.4% 

Identify R-Tech professional development topics 

5.5% 

10.4% 

Provide technical support 

5.2% 

8.0% 


Sources: R-Tech Teacher Survey, fall 2008 and spring 2009. 

Note. Summed percentages represent total of two response categories: (1) the percentage of respondents who 
indicated moderate involvement, and (2) the percentage of respondents who indicated substantial involvement in 
activities. 

In focus group discussions conducted during spring site visits, teachers spoke of their roles in R-Tech. 
Teachers in schools where R-Tech was implemented as part of the regular school day described notably 
greater involvement in R-Tech than teachers in schools implementing supplemental programs. In districts 
that used R-Tech funds to expand access to computer labs, teachers said they scheduled class time in labs 
to use R-Tech resources and used R-Tech data reports to monitor student progress. In addition, teachers 
sent individual students to the lab to get additional support during class time. “I see the teacher. . .in full 
control,” explained one such teacher. “It’s my responsibility to determine what the student needs and how 
to achieve it.” A teacher in another district focus group expressed a similar view: 

I have had some students in my class that were not on [grade] level, so [the students and I] have 
gone and worked on [R-Tech in the lab] to try and fill in missing gaps, and used a different grade 
level than what the student is in. 

A program facilitator noted that teachers used R-Tech services increasingly as TAKS test dates 
approached, explaining, “Sometimes [teachers] bring entire classes down [to the computer lab] just to do 
[remediation programs]; especially right before the TAKS test.” 

In contrast, teachers in focus groups in schools using R-Tech to provide a supplemental instruction 
outside of regularly scheduled classes were less likely to know about R-Tech resources or to report 
participation in the program. When asked about the challenges to teachers’ use of R-Tech resources, a 
teacher in one such focus group said “I didn’t even know there was a program, so no [I don’t experience 
challenges].” A principal in another district implementing a supplemental program explained: 

I think if you talked to teachers they would be maybe unaware of what was available as far as [the 
R-Tech] programs that are available down there [computer lab] for them to use. . .1 don’t think 
everybody completely understands how it can be used. 
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Personal Education Plans 

As part of participation in R-Tech, districts arc required to develop PEPs that enable districts to track 
individual student progress and assess achievement gains (TEA, 2008b). Results from surveys and site 
visits suggest that PEPs were weakly implemented in many districts during R-Tech’ s first year. Less than 
half of the principals (44%) and program facilitators (46%) responding to the spring survey reported that 
they had a moderate or substantial role in developing PEPs (see Figure 3.1). Similarly, less than 14% of 
surveyed teachers indicated a moderate or substantial role in developing PEPs, and less than 20% had a 
moderate or substantial in monitoring PEPs on both the fall and spring surveys (see Table 3.5). 

Further, most principals, program facilitators, and teachers who participated in site visit interviews and 
focus groups were unaware of PEPs and their objectives. Focus group participants could not identify how 
PEPs were developed or monitored. In one focus group, teachers thought PEPs were Individualized 
Education Plans (IEPs) used to guide the instruction of special education students. 2 In several districts, 
interview respondents reported that PEPs were developed by “someone else.” For example, teachers in 
one district said the program facilitator developed PEPs. However, the facilitator said PEPs were the 
responsibility of school administrators, and school administrators said counselors developed PEPs. 3 
Although PEPs may not have been widely used in first year implementation, interview respondents in 
most districts said that R-Tech student outcomes were monitored. Schools relied on vendor-provided 
assessments and student achievement data, as well as traditional forms of assessment, such as grades and 
TAKS scores. 

R-Tech Goals 

The survey of principals and R-Tech facilitators asked respondents to rate their levels of agreement with 
statements about R-Tech goals. Researchers coded responses to emphasize variation between levels of 
agreement, as (-10) strongly disagreed , (-5) disagreed , (0) unsure, (5) agreed, or (10) strongly agreed. 
Table 3.6 presents principals’ and facilitators’ average responses across the fall and spring survey 
administrations. Values closer to 10 indicate higher levels of agreement and values closer to -10 indicate 
higher levels of disagreement. 

On average, principals and facilitators agreed with every statement, indicating positive perceptions of 
R-Tech services. However, mean ratings decreased one point or more across semesters. The greatest 
variation between the fall (5.9) and spring (3.9) semesters was in response to the statement, “R-Tech is 
positively affecting student achievement.” The decline in levels of agreement may reflect increased 
awareness of program outcomes, as well as experience with implementation challenges. 


2 IEPs are used to structure instructional goals and support for students receiving special education services. PEPs are 
used to plan instruction for students participating in R-Tech. 

3 Site visits did not include interviews with school counselors. 
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Table 3.6 

Principals’ and Facilitators’ Perceptions of R-Tech Goals, as a Mean Rating: Fall 2008 
and Spring 2009 


Statement 

Fall 2008 
(N=153) 

Spring 2009 
(N=136) 

Overall, I am pleased with R-Tech services 

5.6 

4.6 

Vendor services align with TEKS/TAKS 

5.7 

4.5 

Vendor services align with campus goals 

5.2 

4.1 

R-Tech is positively affecting instruction on this campus 

4.9 

3.9 

R-Tech is positively affecting student achievement 

5.9 

3.9 

Goals are clear 

5.0 

3.8 

Expectations are clear 

4.7 

3.5 


Sources: R-Tech Principal/Facilitator Survey, fall 2008, spring 2009. 

Note. Number of respondents (N) represents the number of principals and facilitators responding to the 
surveys. Mean ratings based on a 4-point scale: (-10) strongly disagreed , (-5) disagreed , ( 0 ) unsure (5) 
agreed , (10) strongly agreed , with higher ratings indicating greater agreement. 

CHALLENGES TO R-TECH IMPLEMENTATION 

Surveys and site visit interviews also sought to understand the challenges that districts may experience in 
implementing R-Tech. The principal and facilitator survey provided respondents with a list of common 
implementation challenges asking whether issues were not a challenge, a minor challenge, a moderate 
challenge, or a substantial challenge. Table 3.7 presents the summed percentages of survey respondents 
who indicated that challenges created moderate or substantial barriers to R-Tech implementation. 
Summed percentages represent the percentage of respondents indicating banders created a moderate 
challenge to implementation plus the percentage of respondents indicating barriers created a substantial 
challenge. (Supplemental Table A.5 in Appendix A presents the full range of responses for each survey 
item across both survey administrations.) Results indicate that nearly half (49%) of survey respondents in 
spring 2009 felt that the communication of R-Tech goals to parents created challenges. More than a third 
of spring respondents indicated that project reporting requirements (40%), insufficient planning time 
(39%), communicating about R-Tech to staff (38%), and the development of PEPs (37%) were barriers to 
implementation. Notably, the percentage of survey respondents indicating implementation challenges 
increased from fall to spring. This finding likely results from greater implementation of R-Tech in spring 
2009, as well as principals’ and facilitators’ increased experience with the program and its reporting 
requirements. In an open-ended response to the spring survey one principal explained, “We lacked 
adequate staff to fully implement [the] program as we envisioned.... [I] pulled... staff members [from other 
content areas] to make up for the lack of staffing, but this adversely impacted other programs.” 
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Table 3.7 

Moderate and Substantial Challenges to R-Tech Implementation, as a Summed 
Percentage of Respondents: Fall 2008 and Spring 2009 


Challenge 

Lall 2008 
(N=153) 

Spring 2009 
(N=136) 

Communication of R-Tech goals to parents 

32.7% 

48.5% 

Project reporting requirements 

30.7% 

39.7% 

Insufficient planning time 

NA 

39.0% 

Communication of R-Tech goals to staff 

20.3% 

38.2% 

Development of students’ PEPs 

32.7% 

36.7% 

Monitoring students’ progress 

24.9% 

33.1% 

Coordinating training for staff 

26.2% 

31.7% 

Level of technology resources 

19.6% 

27.3% 

Conflicts with other programs 

NA 

24.3% 

Level of technical support 

15.7% 

23.5% 


Sources: R-Tech Principal/Facilitator Survey, fall 2008, spring 2009. 

Note. Summed percentages represent total of two response categories: (1) the percentage of 
respondents who indicated barriers presented a moderate challenge, and (2) the percentage of 
respondents who indicated barriers presented a substantial challenge. NA= Not applicable; this 
item was not included for fall 2008 survey administration. 

Site visit interviews with principals and program facilitators, as well as focus group discussions with 
teachers provide more information about the challenges to implementing R-Tech during the program’s 
first year. 

Communication with Parents 

In interviews, principals described how they communicated R-Tech goals to parents. Most principals said 
they sent letters home, posted information about the program on school Websites, and communicated 
about the program through e-mail. Two districts communicated information about R-Tech at parent 
orientations held at the beginning of the school year. In one district that sent e-mail requests for parent 
permission to include students in R-Tech, the principal reported that parent response was “low,” noting 
“it’s hard to get parents involved.” In another district, a principal explained that some parents objected to 
R-Tech because it created transportation challenges when students were required to stay after school; 
however, other parents liked that “their kid[s] have to kind of face up and reflect on their day.” 

Insufficient Planning Time 

Interviewed principals and program facilitators commented on the lack of planning time for R-Tech 
implementation. “Because [implementation is] so time consuming, I have had to spend a lot of time 
outside of the school day [addressing other responsibilities],” reported one facilitator. A principal in 
another district noted the lack of “upfront time” to implement R-Tech. In another district, a facilitator 
said, “This whole thing [R-Tech] has happened so very quickly that it’s a wonder any of it works.” 

Technology Resources and Technical Support 

Interview respondents in several districts commented on the lack of compatibility between R-Tech 
software and schools’ “antiquated” hardware. One teacher said that the computers loaded software so 
slowly that by the time the program was accessible, the R-Tech tutorial period was nearly over. The 
teacher explained, “[My computers] are so old that by the time [the vendor software] loads and gets on, 
tutorials are over.” Principals in several districts said they purchased new computers to accommodate 
R-Tech programs, but the number of machines was inadequate to serve all students identified for R-Tech 
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services. Administrators reported that the high costs of replacing hardware dictated the number of 
computers purchased. However, schools that “completely revamp[ed]” their technology systems also 
experienced challenges resulting from weak infrastructure and unreliable Internet connections. As one 
principal explained: 

We have really had a hard time making the transition and getting everything in place. ... I really 
believe that has been one of our greatest hurdles because teachers and students alike really had a 
difficult time with [the system] being so intermittently useable. 

The principal continued, describing how service challenges affected R-Tech implementation and 
participation. “It is difficult for me, as an administrator, to really hold them accountable [for 
implementing R-Tech], with all the issues we’ve been having.” Despite challenges, most interview 
respondents remained positive about the potential of R-Tech, noting that most programs encounter 
implementation challenges, particularly in their first year. One principal explained: 

[The challenge] is just learning how [R-Tech] fits in to what we do here. It's kind of a trial and 
error deal, since this is our first year to be able to do this. . .This summer is going to be the first 
time that we implement our credit recovery [component]. We don’t know how it’s going to work. 
It may be a catastrophe, but I know that without this program, there’s no way it could have been 
done. 

Overcoming Challenges 

The spring 2009 survey also included a list of strategies districts may have used to overcome 
implementation barriers and asked respondents to mark all the approaches they incorporated during 
R-Tech’s first year. Table 3.8 presents the percentage of respondents who indicated that each strategy was 
implemented and shows that most respondents worked in districts that purchased additional computer 
hardware (62%) and software (52%) to support R-Tech implementation. More than half of respondents 
(53%) also reported holding information sessions to communicate R-Tech goals to school staff. 


Table 3.8 

Methods to Overcome Challenges to Implementation, as a Percentage of 
Respondents: Spring 2009 


Method 

Spring 2009 
(N=136) 

Purchased additional computer hardware 

61.8% 

Held information sessions for teachers and staff 

52.9% 

Purchased additional computer software 

51.5% 

Upgraded technology infrastructure 

40.4% 

Purchased additional furnishings (e.g., computer tables and chairs) 

23.5% 

Added staff to manage implementation tasks 

24.3% 

Held information sessions for parents and students 

19.9% 


Source: R-Tech Principal/Facilitator Survey, spring 2009. 

Note. Percentages will not total to 100 because respondents could select more than one method. 


The spring survey also included an open-ended item in which respondents could enter other strategics 
used to overcome implementation challenges. Few respondents (14) entered other strategies. One noted 
that coordination of R-Tech with an after school program facilitated implementation, and several wrote 
that they increased communication efforts with parents and provided incentives for student participation. 
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FACTORS THAT CONTRIBUTE TO SUCCESSFUL IMPLEMENTATION 


The spring 2009 survey of principals and program facilitators contained an open-ended item asking what 
contributed most to their schools’ ability to implement R-Tech, and 66% of respondents (90 individuals) 
entered written comments. Researchers reviewed written comments, categorizing responses by common 
themes. Table 3.9 presents the percentage of respondents indicating factors cited in five or more 
comments. 


Table 3.9 

Factors That Contributed to R-Tech Implementation, as a Percentage 
of Respondents: Spring 2009 


Factor 

Spring 2009 
(N=90) 

Strong administrative support 

20.0% 

Additional funding 

20.0% 

Staff buy-in and support 

16.6% 

Added computer hardware and software 

13.3% 

Designated R-Tech facilitator 

7.7% 

Existing technology resources 

7.7% 


Source : R-Tech Principal/Facilitator Survey, spring 2009. 

Note. N is the percentage of survey respondents who entered written comments. 

Responses will not total to 100. Factors cited by fewer than five respondents are 
not included in the analysis. Some responses may be included in multiple 
categories (e.g., “strong administrative support and grant funding”). 

In addition, site visit interviews with principals and program facilitators also asked about the factors that 
contributed to successful implementation. The sections that follow combine interview responses with the 
survey’s written comments to provide more information about the factors contributing to R-Tech 
implementation. 

Strong Administrative Support 

Many survey respondents wrote that strong support from district-level administrators was central to their 
ability to implement R-Tech. A respondent in one district cited “the commitment of central administrators 
to [R-Tech]” as the factor that most supported implementation. In another district, “A forward thinking 
superintendent,” was credited as an important support. “A willing campus and a supportive central 
office,” wrote a respondent in another district, “with these two things, much can be achieved.” 

Funding 

Survey comments also indicated that grant funding was essential to implementation, as one respondent 
wrote: 


The main thing that allowed us to implement the R-Tech pilot program was the grant funding 
from TEA. Without these funds our project would not exist today. The funding allowed our small 
rural district to put together to [sic] high quality computer labs that are utilized by the entire 
school community. 

Most principals interviewed during site visits said additional funding provided by R-Tech was critical to 
implementation. Principals said that R-Tech funding enabled districts to implement programs that they 
could not have afforded otherwise. Principals also noted the strain of providing matching funds for the 
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grant. A high school principal said the district used high school allotment funding (HSA ) 4 to provide 
transportation for students who missed regularly scheduled bus routes in order to participate in R-Tech 
after school. Another principal noted that the district would discontinue R-Tech when state funding 
expired. 

Staff Buy-in 

Survey respondents also pointed to the importance of staff buy-in for successful implementation of 
R-Tech. One respondent wrote of “the willingness of the counselors and technology teachers on campus 
to go above and beyond their normal responsibilities to help students in this program.” A respondent in 
another district cited “A strong desire to offer students opportunities that were not available to them 
before in our small school. Our teachers want the best for our students and will work hard to see that this 
[R-Tech] happens.” 

Additional Computer Hardware and Software 

Survey respondents also reported that the additional computer hardware and software that districts 
purchased using R-Tech funding were important factors in implementation. In one district: 

R-Tech provided a solution to a problem that [school staff] had been struggling to address. 
Through the grant we were able to acquire the necessary hardware and software programs and 
instructors to implement our credit recovery and credit preservation program. We were able to 
provide students struggling with TAKS more resources to improve their performance. 

A survey respondent in another district wrote, “The purchase of computers helped a lot. Our computers 
were getting old and failing on us. With the new computers we gained reliability to work the program 
[R-Tech].” 

Designated Program Facilitators 

Survey respondents and interviewed principals also noted the importance of having a designated R-Tech 
facilitator to support implementation. Principals said facilitators understood R-Tech software, and 
provided training and support for staff. A facilitator noted that previous experience and familiarity “with 
all aspects of grant programming” made R-Tech grant coordination easier. 

Existing Technology Resources 

Several survey comments indicated that technology resources that existed prior to receiving R-Tech 
funding were important to their schools’ ability to implement the program. Survey respondents cited pre- 
existing computer labs, laptops purchased before applying for R-Tech funding, and established 
infrastructure for technology as strong supports to implementation. 


4 Texas districts receive HSA funding ($275 per student) for students in Grades 6 through 12. HSA funds may be 
used to support efforts to increase the rigor of instruction and to encourage students to pursue postsecondary 
educational opportunities (TEC § 39.114). 
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SUMMARY 


Most R-Tech districts implemented technology-based self-paced programs designed to provide tutoring, 
remediation, and credit recovery to struggling students. Program facilitators had the greatest responsibility 
for implementing R-Tech, and most facilitators received training from software vendors and district 
technology coordinators. Most principals were involved in planning the grant, but had little role in 
R-Tech once programs were in place. Teachers had a notably small role in implementing R-Tech; 
however, differences emerged in site visit discussions with teachers in districts implementing 
supplemental and non-supplemental programs. Teachers who worked in districts that offered R-Tech as a 
supplemental program implemented outside of students’ regularly scheduled classes said they had little 
knowledge of the program or its resources. In contrast, teachers working in programs implemented as paid 
of the school day (i.e., non-supplemental programs) were more likely to report using R-Tech resources to 
support classroom instruction and to prepare students for TAKS testing. 

Most surveyed principals and R-Tech facilitators did not experience challenges in implementing the 
program, but those who did noted that communication of program goals to parents, project reporting 
requirements, and insufficient planning time were the most substantial implementation barriers. In site 
visit interviews, principals and facilitators also described the implementation challenges caused by 
outdated computer equipment and poor technology infrastructure. In some districts, challenges were 
offset by strong administrative support for R-Tech, as well as teacher and staff buy-in to the program. 
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Chapter 4 

R-Tech and Students 


R-Tech is designed to provide rural students with access to “educational opportunities that are not 
commonly found. . .in rural school districts” (TEA, 2008a, p. 3). Such opportunities may include 
technology-based academic tutoring, remediation, and credit recovery programs designed to improve 
academic achievement, as well as technology-based dual credit courses and distance learning 
opportunities that enable rural districts to expand their course offerings. This chapter addresses the level 
of student participation in R-Tech provided educational opportunities (Research Question 2) during the 
program’s first year (May 2008-May 2009), and considers the following questions: 

• How are students identified for R-Tech services? 

• What arc the characteristics of students who participate in R-Tech? 

• How many hours per week do eligible students participate in the R-Tech program? 

In addition, the chapter discusses how students access R-Tech services, the subjects they study using 
R-Tech resources, how districts facilitate access to R-Tech services, as well as the benefits and challenges 
students experience as a result of participating in the program. Chapter 6 addresses how participation in 
R-Tech may have affected students’ 2009 TAKS outcomes. 

DATA SOURCES 

The chapter relies on a range of data sources to address the level of students’ participation in R-Tech and 
to explore the benefits and challenges students experience in accessing services. Quantitative data are 
drawn from TEA’s Student Upload data system for the summer 2008, fall 2008, and spring 2009 
reporting periods, as well as from PEIMS. The chapter incorporates findings from the fall 2008 and spring 
2009 surveys of R-Tech facilitators and principals, teachers on R-Tech campuses, and a spring 2009 
survey of students who received R-Tech services during the program’s first implementation year. 

More information on the survey administration processes, response rates, the characteristics of survey 
respondents, and supplemental tables may be found in the appendices of this report. 1 The chapter also 
includes qualitative data collected during site visits to a sample of eight R-Tech districts in April 2009. 
Site visits included interviews with principals and program facilitators on R-Tech campuses, as well as 
focus group discussions with teachers and students who participated in R-Tech during the 2008-09 school 
year. More detailed information about the identification of districts for site visits, site visit activities, and 
the characteristics of districts’ R-Tech programs is presented in Appendix D. 

STUDENT PARTICIPATION IN R-TECH: SUMMER 2008, FALL 2008, AND SPRING 2009 

Across R-Tech’ s first implementation year (May 2008-May 2009), 115 campuses from 62 Cycle 1 
districts participated in the pilot program. 2 As discussed in chapter 1, R-Tech districts arc required to 


'Appendix A contains information about the online survey of R-Tech principals and program facilitators. Appendix 
B contains information about the teacher survey, and Appendix C contains information about the student survey. 

; ln March 2009, TEA developed a “final campus list” of 140 participating R-Tech campuses from 64 districts. In 
early May 2009, it was determined that 23 campuses from the “final campus list” were alternative campuses or 
juvenile detention centers, kindergarten, elementary, or intermediate campuses, not in TEA’s AskTED directory, or 
campuses that said they were not participating in R-Tech. TEA agreed to exclude 20 of the 23 campuses from the 
“final campus list.” At the end of May, TEA reported that Groesbeck ISD had dropped out of the grant and that 
Barber’s Hill ISD had delayed implementation until the 2009-10 school year and should be excluded from analyses 
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track student participation in grant services and to provide reports to TEA through a data upload system 
hosted by the Agency. Across R-Tech’s first year, Student Upload data were submitted to TEA for the 
summer 2008, fall 2008, and spring 2009 grant reporting periods. The summer 2008 student upload 
contains information about students who participated in R-Tech services during the 2008 summer school 
session, the fall 2008 upload contains information about students participating in R-Tech during the fall 
2008 semester, and the spring 2009 upload contains information about students participating in services 
during the spring 2009 semester. 

Table 4.1 presents information about the number and percentage of campuses and districts submitting 
Student Upload data to TEA for each reporting period, as well as campus-level statistics describing the 
level of student participation for each reporting period. Results indicate that less than half of R-Tech 
Cycle 1 districts (47%) and about one-third of campuses (32%) reported students participating in R-Tech 
during summer 2008. However, 90% of districts and 80% of campuses reported serving students in fall 

2008, and 92% of districts and 86% of campuses served students in spring 2009. The number of students 
receiving R-Tech services also increased across reporting periods. In summer 2008, campuses reported 
serving an average of 37 students, in the fall of 2009 the average number of students served increased to 
97, and in spring 2009 campuses served an average of 129 students in R-Tech. Across campuses, the 
range of students served by R-Tech varied from 3 to 258 in summer 2008, from 1 to 716 in fall 2008, and 
from 1 to 687 in spring 2009. As one might expect, the total number of students participating in R-Tech 
by reporting period 3 increased from 1,370 in summer 2008, to 8,795 in fall 2008, to 12,736 in spring 

2009. 


Table 4.1 

R-Tech District, Campus, and Student Participation: Summer 2008, Fall 2008, and Spring 2009 


Data Upload Characteristic 

Summer 

2008 

Fall 

2008 

Spring 

2009 

Number (percentage) of districts reporting (N=62) 

29 (47%) 

56 (90%) 

57 (92%) 

Number (percentage) of campuses reporting (N= 1 1 5) 

37 (32%) 

92 (80%) 

99 (86%) 

Average number of participants per reporting campus 

37.0 

95.6 

128.7 

Median number of participants per reporting campus 

26.0 

50.0 

80.0 

Minimum number of reported participants per campus 

3.0 

1.0 

1.0 

Maximum number of reported participants per campus 

258.0 

716.0 

687.0 

Total Number of Participants 

1,370 

8,795 

12,736 


Source: Texas Education Agency (TEA) R-Tech Student Upload data: summer 2008, fall 2008, and spring 2009. 


of student outcomes included in the second interim report. This excluded five more campuses and two districts, 
resulting in a revised set of 1 15 campuses from 62 districts. 

3 

Students were considered to be participating if they attended one of the 1 15 campuses in the 62 districts that 
participated in the R-Tech program during the summer 2008, fall 2008, or spring 2009 reporting periods, and their 
reported number of primary instructional hours in R-Tech activities was greater than 0. In addition, in spring 2009, 
students were considered to be participating if their reported primary instructional hours in R-Tech activities was 
greater than 0, or they were from Kirbyville High School, Rusk Junior High School, or Rusk High School. Because 
of technical difficulties at these campuses, a number of students were reported as having 0 instructional hours in 
R-Tech activities. TEA confirmed that these students took part in R-Tech activities. 
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Figure 4.1 presents the percentage of students receiving R-Tech services across the 115 campuses that 
participated in R-Tech during the program’s first implementation year. The figure incorporates 
information on total campus enrollments for students in Grades 6 through 12 included in PEIMS, as well 
as district reports of R-Tech student participation for each reporting period. As indicated in the figure, 3% 
of students in Grade 6 through 12 participated in R-Tech in summer 2008, 21% participated in fall 2008, 
and 30% participated in spring 2009. Seventeen percent of students participated in R-Tech in both the fall 
2008 and spring 2009, and 1% of students received services across all three periods (i.e., summer 2008, 
fall 2008, and spring 2009). 



Figure 4.1. Percentages of students participating in R-Tech services by reporting period. 

Sources: Public Education Information Management System (PEIMS), fall 2007 and fall 2008 snapshot data for the 
students attending the 115 participating campuses; Texas Education Agency (TEA) R-Tech Student Upload data: 
summer 2008, fall 2008, and spring 2009. 

Note. The summer 2008 percentage was based on the number of students in Grades 6 through 12 in the 115 R-Tech 
campuses as of the fall 2007 snapshot. Fall 2008 and spring 2009 percentages were based on the number of students 
in Grades 6 through 12 in the 115 R-Tech campuses as of the fall 2008 snapshot. 
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Student Participation in R-Tech: Average Reported Weekly Hours 

R-Tech requires that districts provide students with access to grant-funded services for a minimum of 10 
hours per week, although there are no minimum requirements for student participation in services. Table 
4.2 presents statistics about the extent of student participation in R-Tech services across implementation 
periods, including the average and median 4 number of hours students received services each week, as well 
as the minimum and maximum number of weekly hours reported in TEA’s data uploads for each 
reporting period. Results indicate that R-Tech participation was most intense during summer school, with 
students receiving services for 8.5 hours on average each week, relative to 3.7 and 3.8 average weekly 
hours in fall 2008 and spring 2009, respectively. In summer, 50% of participants reported 7.5 or fewer 
instructional hours per week, while in both fall and spring, 50% of participants reported 2 or fewer 
instructional hours per week. 


Table 4.2 

The Extent of Student Participation 11 in R-Tech Activities: Summer 2008, Fall 2008, and Spring 2009 


Participation Characteristic 

Summer 2008 
(N=l,370 b ) 

Fall 2008 
(N=8,795 b ) 

Spring 2009 
(N=12,736 c ) 

Average Number of Primary Instructional Hours 

8.50 

3.70 

3.81 

Median Number of Primary Instructional Hours 

7.50 

2.00 

2.00 

Minimum Number of Primary Instructional Hours 

0.25 

0.01 

0.01 

Maximum Number of Primary Instructional Hours 4 

>20 

>20 

>20 


Source: Texas Education Agency (TEA) R-Tech Student Upload data: summer 2008, fall 2008, and spring 2009. 
a Extent of participation was based on the primary instructional hours reported in the summer 2008, fall 2008, and 
spring 2009 student uploads. 

b Students were considered to be participating if their reported primary instructional hours were greater than 0. 
c Students were considered to be participating if their reported primary instructional hours were greater than 0. In 
addition, all students in the spring 2009 student upload from Kirbyville High School, Rusk Junior High School, and 
Rusk High School were considered to be participating in R-Tech activities. 

d The number of reported instructional hours per week for some students exceeded 20 (1.1% of students in summer, 
0.6% in fall and 2.3% in spring). These cases were likely reporting errors. In analyses of the effect of participation 
hours on student outcomes presented in chapter 6, researchers omit students with average weekly usage hours above 
20. Chapter 6 provides more information about the omission of outliers in the student upload data. 


4 The median represents the midpoint in a distribution. In the case of R-Tech, the median is the point at which half of 
reported hours fall below and half of reported hours lie above this value. 
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STUDENT IDENTIFICATION FOR R-TECH SERVICES 


The fall 2008 and spring 2009 surveys of principals and R-Tech facilitators asked respondents how 
students were identified for R-Tech services. Results for both fall and spring survey administrations 
indicate that weak academic performance was the primary reason students participated in R-Tech. 

As presented in Table 4.3, more than half of spring 2009 survey respondents indicated that students were 
identified for R-Tech because of poor TAKS performance (68%), poor grades (58%), and teacher 
referrals (56%). While percentage values across response categories were somewhat higher for fall 2008 
survey respondents, the pattern of responses suggests that students were consistently identified for R-Tech 
because of poor academic performance. 


Table 4.3 

Methods of Student Identification for R-Tech Services, as a Percentage of 
Respondents: Fall 2008 and Spring 2009 


Identifier 

Fall 2008 
(N=153) 

Spring 2009 
(N=136) 

Poor TAKS performance 

82.4% 

68.4% 

Poor grades 

73.2% 

58.1% 

Teacher referral 

63.4% 

55.9% 

Performance on other tests 

46.4% 

42.6% 

Parent/student interest 

54.9% 

33.1% 

Curricular need 

26.1% 

20.6% 

Other 

15.7% 

16.2% 

First-generation college student 

5.2% 

6.6% 


Source: R-Tech Principal/Facilitator Survey, fall 2008, spring 2009. 

Note. Percentages will not total to 100 because principals and facilitators were able to select more 
than one method of student identification. 


Consistent with survey findings, many participants in site visit interviews (i.e., principals, R-Tech 
facilitators, and focus group teachers) said that poor TAKS scores were the primary means of identifying 
students for R-Tech services. “We meet at the beginning of school and look at the TAKS results [from the 
previous years],” explained one set of focus group teachers. “We identify [students for R-Tech] before 
school even begins.” A principal in another district underscored the role of TAKS in identifying students, 
noting “TAKS are hugel ” Interview respondents also said that students were referred to R-Tech because 
of weak grades, poor attendance, prior failures, or disciplinary referrals. 

Several site visit districts, however, had school-wide implementations, in which all students participated 
in R-Tech. “The whole school goes [to the R-Tech lab] — sixth, seventh, and eighth grade go each week,” 
explained one teacher focus group. “Some [students] use it extra and are exposed a little more, but 
everyone goes.” In another district, all students had access to R-Tech, but some students were required to 
receive services. “We allow all of the students,” said the campus R-Tech facilitator, “Everyone is invited 
to participate. Some are nudged... and some are forced.” 

THE CHARACTERISTICS OF STUDENTS PARTICIPATING IN R-TECH 

The sections that follow provide information about the students who participated in R-Tech during the 
program’ s first year - . Results arc presented separately for the 2008 summer session, and for the fall 2008 
and spring 2009 semesters. Analyses consider the demographic characteristics of students who received 
R-Tech services, as well as participation by grade level. 
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Summer 2008 


Table 4.4 presents information about the characteristics of students who participated in R-Tech services, 
relative to students who did not, during summer 2008. 5 Relative to students who did not participate in R- 
Tech, summer 2008 R-Tech participants were more likely to be minority students (50% vs. 36%), from 
economically disadvantaged backgrounds (55% vs. 46%), and characterized as special education students 
(17% vs. 13%). Notably, the proportion of African American students receiving R-Tech services was 
double the proportion of African American students on R-Tech campuses who did not participate in 
services in summer school (18% vs. 9%). The proportion of Hispanic students who received R-Tech 
services in summer school was somewhat higher than students who did not (32% vs. 27%). 


Table 4.4 

Characteristics of R-Tech Participants and Non-Participants: 
Summer 2008 


Student Group 

Participants" 0 

(N=l,370) 

Non- 

Participants 1 ’ 

(N=42,535) 

African American 

17.8% 

9.3% 

Hispanic 

32.4% 

27.0% 

White 

48.7% 

62.7% 

Other 

1.2% 

1.0% 

Female 

46.6% 

48.3% 

Male 

53.4% 

51.7% 

Economically disadvantaged 

55.0% 

45.7% 

Special education 

16.7% 

13.2% 

Limited English proficient 

2.9% 

2.8% 


Sources: Public Education Information Management System (PEIMS) fall 2007 
snapshot data for the students attending the 115 participating campuses; Texas 
Education Agency (TEA) R-Tech Student Upload data: summer 2008. 

“Students were considered to be participating if their reported primary 
instructional hours were greater than 0. 

b Non-participants attended Grades 6 through 12 in the 115 campuses in the 62 
districts that participated in the R-Tech program in 2007-08, but were not 
identified as actually participating in R-Tech activities in the summer 2008 data 
upload. 

c There were 217 of the 1,370 summer 2008 participants who had missing 
demographic information. The percentages in the table were based on the 1,153 
participants who had demographic information. 


^Percentages of participants and non-participants were based on 2007-08 Grades 6 through 12 enrollments in the 1 15 
participating campuses. 
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Fall 2008 and Spring 2009 


Table 4.5 presents the characteristics of students who received R-Tech services, relative to those who did 
not, for fall 2008 and spring 2009 reporting periods. 6 In contrast to students receiving R-Tech services 
during the 2008 summer school, the characteristics of students participating in R-Tech during the regular 
school year were largely the same as non-participants. 


Table 4.5 

The Characteristics of R-Tech Participants and Non-Participants: Fall 2008 and Spring 2009 


Student Group 

Fall 

2008 

Spring 

2009 

Participants' 1 ' 11 

(N=8,795) 

Non- 

Participants 1 ’ 

(N-34,380) 

Participants 2,0,6 

(N=12,736) 

Non- 

Participants 1 ’ 

(N=30,874) 

African American 

8.9% 

9.8% 

8.8% 

9.9% 

Hispanic 

30.5% 

27.3% 

27.1% 

28.2% 

White 

59.5% 

61.9% 

62.9% 

60.9% 

Other 

1.1% 

1.0% 

1.2% 

1.0% 

Female 

47.2% 

48.5% 

47.5% 

48.5% 

Male 

52.8% 

51.5% 

52.5% 

51.5% 

Economically disadvantaged 

51.1% 

47.5% 

52.2% 

46.7% 

Special education 

14.5% 

12.1% 

13.7% 

12.1% 

Limited English proficient 

3.3% 

2.3% 

3.3% 

2.3% 


Sources: Public Education Information Management System (PEIMS) fall 2008 snapshot data for the students 
attending the 115 participating campuses; Texas Education Agency (TEA) R-Tech Student Upload data: fall 
2008 and spring 2009. 

“Students were considered to be participating if their reported primary instructional hours were greater than 0. 
b Non-participants attended Grades 6 to 12 in the 115 campuses in the 62 districts that participated in the R-Tech 
program in 2008-09, but were not identified as actually participating in R-Tech activities in the data uploads. 
“Students were considered to be participating if their reported primary instructional hours were greater than 0. In 
addition, all students in the spring 2009 student upload from Kirbyville High School, Rusk Junior High School, 
and Rusk High School were considered to be participating in R-Tech activities. 

d There were 244 of the 8,795 fall 2008 participants who had missing demographic information. The percentages 
in the table were based on the 8,55 1 participants who had demographic information. 

“There were 676 of the 12,736 spring 2009 participants who had missing demographic information. The 
percentages in the table were based on the 12,060 participants who had demographic information. 


Percentages of participants and non-participants were based on 2008-09 Grades 6 through 12 enrollments in the 1 15 
participating campuses. 
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Grade Levels Served by R-Tech 


Table 4.6 presents information about the percentages of students participating in R-Tech by grade level 
and implementation period (i.e., summer 2008, fall 2008, and spring 2009). For the most part, variations 
in the percentage of student participants by grade were minor across R-Tech’ s first year. Proportionately 
more middle school than high school students (65% vs. 35%) participated in the program as a part of the 
2008 summer session, and a notably large proportion of 8th-grade students participated in services during 
summer school (29%). The proportion of 8th-grade students receiving services in summer school is likely 
a function of districts’ efforts to provide remediation and support to students transitioning to high school 
and to reduce middle school retention rates. 


Table 4.6 

The Percentage of Students Participating in R-Tech by Grade: 
Summer 2008, Fall 2008, and Spring 2009 


Grade Level 

Summer 2008 
(N=l,370 a ) 

Fall 2008 
(N=8,795 a ) 

Spring 2009 
(N=12,736 b ) 

6 

19.4% 

10.8% 

14.0% 

7 

16.6% 

14.5% 

16.0% 

8 

29.3% 

16.4% 

14.7% 

9 

9.2% 

15.3% 

17.6% 

10 

9.9% 

16.8% 

15.3% 

11 

10.7% 

15.2% 

13.3% 

12 

4.8% 

10.9% 

9.2% 

Total 

100.0% 

100.0% 

100.0% 


Sources: Public Education Information Management System (PEIMS) fall 2008 
snapshot data for the students attending the 115 participating campuses; Texas 
Education Agency(TEA) R-Tech Student Upload data: summer 2008, fall 2008, 
and spring 2009. 

“Students were considered to be participating if their reported primary 
instructional hours were greater than 0. 

b Students were considered to be participating if their reported primary 
instructional hours were greater than 0. In addition, all students in the spring 2009 
student upload from Kirbyville High School, Rusk Junior High School, and Rusk 
High School were considered to be participating in R-Tech activities. 

HOW STUDENTS ACCESS R-TECH AND WHAT THEY STUDY USING R-TECH 
RESOURCES 

The sections that follow provide information about how districts provide students with access to R-Tech 
services and the subject areas that students study using R-Tech resources. Findings arc drawn from a 
range of data sources, including the spring 2009 survey of students, the fall 2008 and spring 2009 survey 
of principals and R-Tech facilitators, as well as from information collected during spring 2009 site visit 
interviews with principals and R-Tech facilitators, and focus group discussions with teachers and students 
participating in R-Tech services. 
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Student Access to R-Tech Services 

The spring 2009 survey of students participating in R-Tech asked about the places and times students 
participated in R-Tech, as well as how often they accessed services. Figure 4.2 presents students’ 
responses. Although the frequencies of access vary, results indicate that most respondents participated in 
R-Tech during regular - class time (91% — the sum of respondents indicating that they accessed resources 
rarely, sometimes, often or almost daily), at home using a home computer (79%), at school during a free 
period (62%), and before (60%) or after (56%) school. 


At school during regular class time 
At home using a HOME computer 
At school during a free period 
At school BEFORE classes begin 
At school AFTER classes are over 
At school during lunch 
At a public library 
Other 

At home using a SCHOOL laptop/device 


13% | 18% | 19%~ 

~8% I 13% I ~T7% 





Percentage of Students (N=2,993) 


■ Never DRarely BSometimes DOften BAImost Daily 


Figure 4.2. Students’ access to R-Tech services, as a percentage of respondents: spring 2009. 

Source: R-Tech Student Survey, spring 2009. 

Note. The R-Tech Student Survey included an open-ended item allowing respondents to define the “other” times in 
which they accessed R-Tech services. However, no student provided a written response. 


Information collected during interviews and focus groups conducted as part of the spring 2009 site visits 
to R-Tech districts provide more information about how students participated in R-Tech. The sections that 
follow address the most common ways for students to access R-Tech services. 

Accessing R-Tech during class time. Although R-Tech was intended to provide supplemental 
instruction services, about 40% of R-Tech districts incorporated R-Tech services into regular instruction, 
and 60% of surveyed students reported using R-Tech resources during class time often or almost daily. A 
focus group teacher in a district implementing a self-paced instructional program reported scheduling 2 
days a week in the computer lab to use R-Tech resources because students preferred using computers to 
traditional classroom resources. In another district, teachers took students to the computer lab to use 
R-Tech software at least once a week and incorporated software-reported scores into class grades. “We’re 
the ones that have to boost the students and [let] them know that this is something we’re going to use,” 
explained a focus group teacher. “I think taking grades is very important.” Focus group students in a 
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school implementing a school-wide technology immersion program reported using R-Tech resources 
every day in at least one class. They noted that ELA made the most extensive use of technology 
resources, and that students also used laptops to take notes in other classes. 

Accessing R-Tech from a home computer. More than half of surveyed students (58%) reported that 
they accessed R-Tech resources at home using a home computer often or almost daily. A focus group high 
school student in a district offering a supplemental self-paced tutoring program explained, “I found out I 
could do it at home, and we can do tutoring even after school and before school, but most of the time I do 
it at home because that’s when I have more time.” However, another student in the same focus group said, 
“I mainly use it here [at school] . . .1 don’t have Internet access at home.” Teachers said that home access 
enabled students who had “missed a lot” to catch up. 

Accessing R-Tech at school during a free period. A third (33%) of surveyed students reported using 
R-Tech resources during a free period often or almost daily. In focus groups, some teachers said they 
assigned students to use R-Tech resources during study hall or during in-school suspension. A principal in 
a district that had used R-Tech resources to employ a full-time computer lab facilitator said staffing was 
key to providing student access throughout the school day because prior to R-Tech, the lab was closed to 
students outside of class. “It all has to do with the staffing of it,” explained the principal. “We had the 
computer lab before, but [students] could only go in there if [their] class was going in there.” 

Accessing R-Tech before or after school. Smaller percentages of surveyed students indicated that they 
accessed R-Tech services often or almost daily before (28%) or after (23%) school. In interviews, R-Tech 
facilitators noted the challenges of providing services outside of the regular school day. One facilitator 
explained, “From the kids’ side, I think there were a lot of activities that kept them from participating. . . 
They [students] would even express that they needed more time in their lives.” In several districts, 
interview respondents noted the challenge of providing services before or after school for rural students 
who must take buses to get home. “Transportation is a big issue here,” explained one set of focus group 
teachers, “because our students can’t walk home. They get here on the bus at quarter till [time school 
begins] and there’s no time for extra help in the mornings.” Some districts adjusted bus schedules and 
permitted students to eat breakfast in R-Tech labs in order to offset transportation and time limitations. 

What Students Study Using R-Tech Resources 

The student survey also asked what subject areas students addressed using R-Tech resources, and Figure 
4.3 presents the percentage of students who responded to each subject area for each level of schooling and 
for all students. As indicated by the figure, middle school students were more likely to focus on math and 
science, while high school students were more likely to focus on reading /EFA, languages other than 
English, and other subject areas. Of the students who entered written responses clarifying “other” subject 
areas, high school students were most likely to report using R-Tech to focus on Business Computers and 
Information Systems (BCIS) coursework (55 responses) or career and technical education fields, such as 
auto mechanics (11 responses), agriculture (5 responses), or industrial technology (5 responses). Middle 
school students were more likely to indicate that they studied all subjects (13 responses), as well as 
keyboarding and computer skills (10 responses). 
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Figure 4.3. The subject areas students study using R-Tech resources, as a percentage of survey 
respondents: spring 2009. 

Source: R-Tech Student Survey, spring 2009. 

In focus group discussions, many students commented that R-Tech content was focused on TAKS 
preparation in core subject areas. As one student explained, R-Tech “address[es] the objectives that you 
didn’t get on TAKS.” Focus group students also said that R-Tech programs focused on review, rather 
than introducing new content. One focus group student explained, “[R-Tech] helps you review what you 
already know or get you ready for what you are going to learn.” A student in another district explained 
that R-Tech addresses “stuff you already know but need help with.” 

FACILITATING STUDENT PARTICIPATION IN R-TECH 

As noted in the previous section, some students experienced challenges in participating in R-Tech 
because of conflicts with other activities and transportation issues. This section addresses the barriers that 
may limit students’ ability to participate in R-Tech services, as well as the strategies districts implemented 
to overcome banters. Findings arc drawn from the fall 2008 and spring 2009 survey of principals and 
R-Tech facilitators, site visit interviews with principals and facilitators, as well as focus group discussions 
with teachers and R-Tech students. 

Barriers to Student Participation 

The fall 2008 and spring 2009 surveys of principals and R-Tech facilitators presented a list of potential 
banters to student participation in R-Tech and asked survey respondents whether issues created a minor, 
moderate, or substantial barrier to students’ participation or whether issues were not a barrier. Across 
both survey administrations, most principals and facilitators (more than 60% of respondents) indicated 
that each issue was either not a barrier or a minor barrier. Table 4.8 presents the summed percentages of 
survey respondents who indicated each issue presented a moderate or substantial banier to student 
participation in R-Tech for each survey administration. Summed percentages represent the percentage of 
respondents indicating barriers created a moderate challenge to students’ participation plus the percentage 
of respondents indicating banders created a substantial challenge. (See supplemental Table A.6 in 
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Appendix A for individual percentages by survey item, barrier rating, and survey administration.) The 
sections that follow provide more information about barriers drawn from site visit interviews and focus 
groups. 


Table 4.8 

Moderate and Substantial Barriers to Student Participation in R-Tech Services, as a Summed 
Percentage of Respondents: Fall 2008 and Spring 2009 


Moderate/Substantial Barriers (summed percentages) 

Fall 2008 
(N=153) 

Spring 2009 
(N=136) 

Student resistance 

30.7% 

35.3% 

Conflicts with athletic programs 

24.2% 

30.9% 

Transportation 

31.4% 

27.2% 

Conflicts with school-sponsored extra-curricular activities 

23.6% 

26.5% 

Conflicts with non-school extra-curricular activities 

12.4% 

20.6% 

Conflicts with student employment 

24.2% 

20.6% 

Parent resistance 

11.2% 

8.0% 

Other 

24.0% 

36.3% 


Source : R-Tech Principal/Facilitator Survey, fall 2008, spring 2009. 

Note. Summed percentages represent total of two response categories: (1) the percentage of respondents who 
indicated barriers created moderate challenges, and (2) the percentage of respondents who indicated barriers 
created substantial challenges. 

Student resistance. Principals and facilitators considered “student resistance” to be the most common 
barrier to participation in spring 2009, and the increase in the percentage of respondents indicating that 
resistance created a moderate or substantial barrier from fall to spring (31% to 35%) suggests that student 
resistance may have increased over the program’s first year. During site visit interviews, teachers and 
R-Tech facilitators explained the challenges created by student resistance. “Sometimes kids just flat 
refuse to come,” noted one focus group teacher in a high school implementing a supplemental tutoring 
program. Teachers in another district offering self-paced tutoring said students felt there was a negative 
social stigma to staying after school. “I think some of it [resistance] too, is it’s not cool [to stay after 
school for tutoring],” explained one teacher. In another district, teachers said that students were bored 
with the program. 

Conflicts with athletic programs. Similar to results for student resistance, the percentage of principals 
and R-Tech facilitators indicating that conflicts with athletic programs created banders to also increased 
from fall to spring (24% to 31%), which may reflect increased awareness of challenges as more students 
participated in seasonal sports programs. In site visit interviews, however, only one R-Tech facilitator 
noted that sports programs “get in their [students] way” of participating in the program. “From the kids’ 
side,” explained a principal, “I think there [are] a lot of activities that [keep] them from participating.” 

Transportation. As noted in a previous section, bus schedules and transportation issues created 
challenges for students in arriving early or staying after school to use R-Tech resources. Flowever, unlike 
student resistance and conflicts with athletic programs, the percentage of principals and facilitators 
indicating that transportation created moderate or substantial banters decreased from the fall to the spring 
survey administration (31% to 27%), which suggests that districts were able to resolve transportation 
issues to some extent. Transportation challenges were a common theme across site visit districts and 
interview and focus group respondents. A facilitator explained “We have kids say ‘If I don’t go home 
right now, I don’t have a ride home.’” A principal in another district said that parents were unwilling to 
provide transportation for students assigned to R-Tech, noting “Parent support isn’t there, so it’s [R-Tech] 
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not a priority at the family level, and if it’s not a priority there, it’s not a priority for the child.” To offset 
transportation challenges, several districts added late bus routes for R-Tech students. 

Other challenges. Survey responses also indicated that “other” challenges to student participation 
increased over the program’s first implementation year (24% to 36%). Site visit interviews and focus 
group discussions provided more information about the range of challenges students encountered. 
Teachers in several districts spoke of students’ family responsibilities as barriers to program participation. 
“Home life,” said one teacher. “Some [students] have to get home to take care of their brother, sister, or 
their parents in the afternoon.” Teachers in another district noted, “Absenteeism is always a problem [for 
R-Tech] — jobs, rides sometimes, family responsibilities when [students] are taking care of [their] 
siblings.” To accommodate student responsibilities, one school shifted after school tutoring to the 
morning, but the change created conflicts for students receiving free- and reduced-price breakfasts. As a 
compromise, the school permitted students to bring their breakfasts into the computer lab. Teachers in 
several districts said that students’ lack of proficiency using technology and poor school infrastructure 
also created barriers to students’ participation. 

Overcoming Barriers 

The spring 2009 principal and R-Tech facilitator survey also asked respondents about the strategies they 
implemented to overcome banters to student participation. Figure 4.4 presents the percentage of 
respondents indicating that their schools implemented strategics designed to improve student 
participation. Results indicate that about half of respondents worked in districts that expanded access 
times and locations (51%) or required participation for some students (49%). Smaller percentages of 
respondents indicated that their districts implemented incentives for participation (24%), developed 
R-Tech promotional materials (18%), or added transportation (18%). Survey respondents who entered 
written responses describing “other” strategies for overcoming banters described implementing R-Tech 
as part of Saturday school, enabling access to R-Tech resources at a public library, and publishing a 
monthly newsletter describing the R-Tech program and including photos of students who experienced 
success as a result of program participation. 
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Expanded available times and locations 


51% 


Required participation for some students | 49% 

Implemented incentives for participation 24% 

Developed promotional materials 18% 

Additional transportation 18% 

Other 12% 

0% 10% 20% 30% 40% 50% 60% 

Percentage of Respondents (ISM 36) 

Figure 4.4. Strategies for overcoming barriers to student participation: spring 2009. 

Source : R-Tech Principal/Facilitator Survey, spring 2009. 

Note. Percentages will not total to 100; respondents were able to indicate multiple strategies. 

Site visit interviews and focus group discussions provided more detailed information about how districts 
overcame participation barriers. In an attempt to overcome the negative social stigma attached to R-Tech, 
one school allowed students to bring friends to the lab. “If they can recruit a friend to come [with them to 
the lab], that helps,” explained the principal. The school also provided snacks to “entice” students to 
participate. One principal reported using “propaganda,” to encourage student participation, “I tell them 
it’s filling up so they have to hurry if they want to get in.” Teachers in another district said they pushed 
students to participate. “We are dedicated to the kids,” explained one focus group teacher, “and we stay 
after them to make sure [they participate].” 

To offset students’ lack of proficiency using technology resources, one R-Tech facilitator labeled 
computer lab equipment and posted instructions detailing how to use each resource. Schools that 
experienced infrastructure challenges worked closely with district technology specialists to address 
system barriers, such as unreliable Internet connections and compatibility issues between old and new 
hardware and software. Teachers in one focus group described infrastructure challenges as “growing 
pains” that will be “worked out with time.” 


60 




THE BENEFITS OF STUDENT PARTICIPATION IN R-TECH 


Students participating in the spring 2009 survey responded to a list of statements describing the benefits 
they may have enjoyed as a result of participation in R-Tech, indicating their level of agreement with each 
statement. Researchers coded their responses: strongly disagreed (-10), disagreed (-5), unsure (0), agreed 
(5), and strongly agreed (10) as a means to clearly illustrate variations in students’ levels of agreement. 
Table 4.9 presents students’ mean responses, sorted in terms of the “All Respondents” column. Values 
closer to 10 indicate higher levels of agreement and values closer to -10 indicate higher levels of 
disagreement. 


Table 4.9 

Students’ Level of Agreement with Statements about the Benefits of R-Tech Participation, as a 
Mean of Respondents: Spring 2009 


Statement 

Middle 
School 
(n= 1,530) 

High 
School 
(n= 1,463) 

All 

Respondents 

(N=2,993) 

Technology resources allow me to work at my own pace. 

3.9 

4.6 

4.2 

Technology resources make learning more interesting. 

3.8 

3.3 

3.5 

I learn more when I use technology resources. 

3.3 

3.4 

3.4 

Technology resources allow me to focus on the areas where 
I need extra help. 

3.2 

2.8 

3.0 

I feel more confident about my school work since I started 
using technology resources to help me learn. 

2.2 

2.0 

2.1 

When I use computers to learn, I know right away whether 
I got a question right or wrong. 

1.6 

1.8 

1.7 

My grades have improved since I began using technology 
resources for learning. 

1.5 

1.2 

1.4 

Technology resources have allowed me to make up 
coursework I have missed. 

0.0 

1.5 

0.7 

Technology resources allow me to take classes taught by 
teachers who are NOT at my school. 

-2.4 

0.3 

-1.1 


Source'. R-Tech Student Survey, spring 2009. 

Note. Items have been coded: strongly disagree (-10), disagree (-5), unsure (0), agree (5), strongly agree (10). 
Items closer to 10 indicate higher levels of agreement. 


The student survey also included an open-ended item in which students could write about the aspects of 
the program that they valued most. In addition, site visit interviews and focus group discussions also 
addressed the benefits students may receive from participating in R-Tech. The sections that follow 
combine sources to discuss the range of benefits students experienced from program participation. 

Self-Paced Programs 

Students agreed most strongly that technology resources allowed them to work at their own pace. In open- 
ended survey comments a student explained, “You don’t have to wait on other people to catch up to you 
or work slower for the sake of other students.” Other students appreciated that there was “less pressure” to 
work quickly with technology than in the traditional classroom. “I can go at my own pace,” said a focus 
group student. “It feels like one-on-one tutoring, so nobody can bother me.” In another focus group 
students said they learned more when they were able to work at their own pace. “It helps me learn at a 
pace [that is] best for me,” noted a middle school student. “I can concentrate more and I am able to 
understand.” Another student agreed, “If you learn at your own pace, it stays in your brain longer.” 
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Interesting Programs 

Students also agreed with the statement indicating that technology-based instruction is more entertaining 
than traditional classroom instruction. In focus groups discussions, many students said they enjoyed 
games and other “fun” ways to learn concepts included in R-Tech software programs. However, many 
students were excited simply to have access to technology, including computers, laptops, and iPods. “You 
actually want to be on [the computer],” said one focus group student, “instead of just doing a packet [of 
worksheets].” 

Improved Learning 

Students also said they learned more when they used technology. “When you listen to a teacher 
sometimes they go too fast, or too slow” wrote one student in an open-ended comment. “I won’t have to 
raise my hand and disrupt the class to learn something simple,” wrote another. “It’ s like my own personal 
teacher.” Teachers and facilitators said that the individualized instruction provided by R-Tech 
accommodated different learning styles. In a district implementing iPods with content targeted to ELL 
students a facilitator reported that one student benefited from the auditory component of iPods. “He can’t 
just see it [lesson content], he has to hear it,” explained the facilitator. In one focus group, students felt 
that R-Tech required them to be “held more responsible” and “learn to think on [their] own.” In addition 
to the responsibility of self-directed learning, some students said that R-Tech instruction was “more 
demanding” than learning in a traditional classroom setting. 

Focus on Areas of Instructional Need 

Locus group students said that R-Tech allowed them to focus on the specific areas in which they needed 
remediation and support. “I usually use it for math for TAKS preparation because I am doing very poorly 
in math,” said a focus group student. “[R-Tech software] gives you a quiz and then gives you a whole list 
of what you struggled on and what you need help with the most,” explained a student in another district, 
“and then you click on that [skill] and work on it.” In another district, students appreciated that R-Tech 
resources facilitated review. “You can go back over it [the lesson], and if you need help you can ask the 
teacher,” a student explained. 

Increased Confidence 

Students’ survey responses also indicated that R-Tech has increased their confidence, and teachers 
agreed. One teacher explained that students who participated in R-Tech were “feeling better about 
themselves and feeling more confident.” High school teachers in another district told of a student who 
was able to bring up failing grades and pass the exit-level TAKS as a result of participation in R-Tech. 
Locus group teachers noted, “[R-Tech] increased his confidence level to the point that he is now planning 
on attending college . . .and he wasn’t.” In an open-ended survey comment, one student wrote, “I feel that 
I am part of the learning process and my opinion matters. I have a voice.” 

Immediate Feedback 

Students also agreed that technology resources supported learning by providing immediate feedback. 
“When I get something wrong, the computer explains what the correct answer is and tells me exactly what 
I did wrong and how to correct it,” said a focus group student. “Teachers don’t usually know that.” 
Another student in another district said, “I know if I get things right or wrong when I answer them. . .with 
explanations, instead of waiting on a teacher to grade it and maybe explain why the incorrect answers 
were [incorrect].” 
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Improved Grades 

While surveyed students were less likely to agree that participation in R-Tech had improved their grades, 
teachers and program facilitators on R-Tech campuses attributed improved grades to the program. 
“[Students] are doing better in their class work because it [R-Tech] reinforces what they have learned in 
class and gives them more confidence,” noted one facilitator. Some focus group students also credited 
improvements in grades to participation in R-Tech. “I brought my grades up,” explained one student. “I 
thought my grades would never be this good, until they put me in this system [R-Tech].” 

Missing Coursework 

Although surveyed high school students were more likely to agree that R-Tech had enabled them to 
recover course credits than middle school students (1.5 vs. 0.0), respondents from both middle and high 
schools cited the benefit of recovered credits during site visit interviews. For example, a high school 
teacher told of a student who graduated because of participation in R-Tech: 

We had a senior this year that would not have passed math without it [R-Tech]. Being here 
[computer lab] with just the computer, he did not feel the intimidation factor that he felt in a 
classroom situation and he would still bring things to ask me, but nobody else knew that he was 
asking. I really think without that [support] he wouldn’t have [graduated]. 

A facilitator in another district that implemented a middle school summer school program described how 
R-Tech participation helped students who had failed courses: 

It [R-Tech] allowed us to have a summer program that we would not have had otherwise. . . which 
[enabled] some middle school students who would not have passed — they would not have been 
able to go on to high school — but they were able to go to high school because they attended the 
summer program. 

Other Benefits of R-Tech Participation 

Convenience. In open-ended survey comments, students expressed appreciation for the convenience 
technology provides, indicating it “makes work quicker and easier.” For example, students who had 
laptops no longer had to carry books and materials to class because their work was saved on the 
computers. Many students said that using computers enabled them to better organize their work. Several 
surveyed students wrote that they preferred using the Internet to reference books. “[The Internet] allows 
me to find answers to questions faster,” explained one focus group student. “It is much faster than looking 
things up in a book.” Another student reported that writing essays on the computer “is much easier than 
writing with a pencil or pen and [Spell Check] also helps you with your spelling.” 

Expanded access to information. The Internet enables students to connect with communities around the 
world and expands access to information to students in remote settings, and these benefits were not lost 
on R-Tech students. One surveyed student wrote, “I learned to do new things I [had] never heard of.” A 
focus group student agreed, noting “[Internet access] helps you to get more experienced in different fields 
of learning” and provides information about new topics “when other people can’t.” In another focus group 
a student explained, “[The Internet] provides. . .information that the teacher. . .may not be able to give 
you.” 

Increased proficiency using technology. Participants in site visit interviews and focus groups also 
pointed to students’ improved proficiency using computer resources as a benefit of R-Tech participation. 
“Students who didn’t know how to use computers before now have been exposed richly,” noted one 
principal. Several students commented that participation in R-Tech had increased their skills in using 
technology, as one student wrote, “[the program enabled students to] get better at using computers and 
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other equipment.” Focus group students also cited benefits from increased exposure to technology. 
Students in a technology immersion program said that daily use of a laptop to take notes and complete 
assignments was ‘“good practice” for college. In another district, students said that increased proficiency 
using technology might lead to ‘“promotions” in future employment. 

THE CHALLENGES STUDENTS EXPERIENCE PARTICIPATING IN R-TECH 

The student survey also asked about the challenges students experienced in participating in R-Tech. The 
survey contained a list of common program challenges and an open-ended item for students to enter 
written responses addressing what they disliked about the program. Researchers used the same approach 
to illustrating students’ level of agreement, coding responses: strongly disagreed (-10), disagreed (-5), 
unsure (0), agreed (5), and strongly agreed (10). Table 4.10 presents students’ average responses, sorted 
in terms of the ‘“All Respondents” column. Again, values closer to 10 indicate higher levels of agreement 
and values closer to -10 indicate higher levels of disagreement. 


Table 4.10 

Students’ Level of Agreement with Statements about the Challenges of R-Tech Participation, as a 
Mean of Respondents: Spring 2009 


Challenges 

Middle 
School 
(n= 1,530) 

High 
School 
(n= 1,463) 

All 

Respondents 

(N=2,993) 

Computer and other technology resources at my school are 
often slow or broken. 

-0.6 

0.5 

-0.1 

My school’s Internet connections are too slow or are often 
not working. 

-0.6 

-0.3 

-0.5 

I have difficulty arranging to come to school early or to 
stay after school to use technology resources. 

-1.3 

0.0 

-0.7 

My teacher can’t fix things when something is wrong with 
the technology resources I use for learning. 

-1.7 

-1.1 

-1.4 

I am bored by the school work I do using technology 
resources. 

-1.5 

-1.2 

-1.4 

Using my school’s technology resources sometimes 
interferes with my extra-curricular activities. 

-2.1 

-1.9 

-2.0 

The programs I need are not on the computer. 

-3.4 

-2.1 

-2.8 

I have trouble getting my questions answered when I use 
technology resources. 

-3.4 

-2.5 

-3.0 


Source'. R-Tech Student Survey, spring 2009. 

Note. Items have been coded: strongly disagree (-10), disagree (-5), unsure (0), agree (5), strongly agree (10). 
Items closer to 10 indicate higher levels of agreement with each statement. 


Across survey items, averaged responses indicate that most students did not experience challenges 
participating in R-Tech; however, many of these themes emerged in site visit discussions and in students’ 
written responses to the open-ended survey item asking what they ‘“like least about learning with 
technology resources.” The sections follow provide more information about the challenges students 
described. 

System Challenges 

While surveyed students expressed mild levels of disagreement, on average, with survey items stating 
school technology resources (-0.1) and Internet connections (-0.5) “are often slow or broken,” some 
students entered written comments describing the challenges they experienced in working with unreliable 
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Internet connections and outdated computer hardware. “You never know when the technology isn’t going 
to work and you have something important you need to get done,” wrote one such student. Several 
surveyed students wrote comments noting teachers often assigned R-Tech assignments as homework 
when school Internet connections failed, which created challenges for students who did not have home 
computers or home Internet access. Students also commented that they were unable to “lock” work they 
saved on school computers, and sometimes other students would tamper with their files. Some students 
complained of lost work when software or school networks failed. “Sometimes my computer quits, or an 
application quits, unexpectedly and my [information] is lost,” wrote one surveyed student. Unreliable 
school technology resources also created challenges for students enrolled in dual credit courses. Students 
wrote that video images often froze during live lectures, which caused them to miss important information 
and affected their course grades. 

During focus group discussions conducted as part of site visits, students provided more information about 
the challenges created by technology failures. Students in several focus groups described frustration 
accessing R-Tech resources. In addition to Internet failures and unreliable school networks, students 
explained that many school computers were outdated and took a long time to load R-Tech programs. 
Students in a district implementing a school-wide immersion program reported frequent system crashes 
that resulted because the district lacked sufficient infrastructure to support many students accessing the 
Internet at the same time. 

Lack of Technical Experience 

Students entering written comments on the spring survey also reported that teachers’ inexperience using 
technology, as well as their own inexperience with resources created challenges to R-Tech participation. 

“I hate computers,” wrote one survey respondent. “They are hard to work on.” One student who lacked 
experience using technology wrote “it takes longer using technology resources to get the task done.” 
Another student was frustrated by “the lack of [explanations] of some terms and buttons.” 

Students also wrote that teachers lacked expertise using computers. Focus group students in one district 
explained “it’s frustrating” for students with technical questions about computers “because the teachers 
don’t know what to do, so they just send [students] to another computer.” A focus group student in the 
district implementing a technology immersion program said, “I hate when teachers want us to do things 
on our laptops that aren’t possible since they do not know a whole lot about some of the programs.” 

Challenges Using R-Tech Software 

Students also reported that R-Tech software programs did not meet their needs. One survey respondent 
wrote, “[The programs] do not help me at all.” Focus group students in a district implementing a TAKS 
remediation program said that software was not aligned with the format of the TAKS test; that the 
software required fill-in-the-blank responses while TAKS was a multiple choice test. “I don’t like that it 
gives you more than one guess,” said a focus group student in the same program, “You don’t get that 
many guesses on the real test.” 

In another district, focus group students said R-Tech software did not provide sufficient instruction in key 
concepts or clear information about incorrect answers. One student explained: 

[At the end of the program], it will say [the question number] you got wrong, but it won’t tell you 
what the question was or what was [the] right [answer]. So you don’t know what the answer is. 

So you just have to keep doing it over and over again. 

Focus group students in another district experienced similar challenges. One student said that R-Tech 
lessons “aren’t clear enough. . .and don’t give you enough resources to figure it out, so you still have 
questions on it.” District teachers who participated in a focus group agreed. “Sometimes the explanation it 
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[the software] gives is not quite as clear as it could be,” noted one teacher. “Sometimes the lessons arc not 
complete enough for them [students] to do independent [work].” Focus group students in another district 
that implemented R-Tech as a reading remediation program said that while the software adjusted content 
for students’ reading levels, the vocabulary was often too challenging. The program highlighted new 
vocabulary and provided definitions for highlighted words, but a focus group student explained, “Not all 
the hai'd words are highlighted, so you can’t get the definition... You’re just guessing.” 

General Disinterest 

As discussed earlier in this chapter, R-Tech facilitators and principals who responded to surveys indicated 
that student disinterest was a key challenge for R-Tech, and many students who participated in site visit 
focus groups confirmed this understanding. One focus group student who participated in a reading 
remediation program noted, “All you are reading is fact articles and a lot of kids read better with fiction.” 
Students in another district focus group described R-Tech as “real boring” and “real dull.” To overcome 
boredom, some focus group students reported using R-Tech resources to browse the Internet and listen to 
music online, and some focus group students said, if given the choice, they would not participate in 
R-Tech again. 

To overcome student resistance, some districts required students to receive R-Tech services. A student 
responding to the online survey noted, “I dislike when teachers make [services] mandatory instead of 
optional.” Another district assigned struggling students to receive R-Tech services during the regular 
school day, and scheduled R-Tech during students’ elective periods, which frustrated some students. “I 
want to go to art class,” explained one focus group student, “not just sit around and do [the R-Tech 
program] all day.” 

SUMMARY 

The analyses presented in this chapter indicate that most students were identified for R-Tech services 
because of weak academic performance (e.g., low TAKS scores and poor grades). Students who 
responded to the evaluation’s spring 2009 survey reported that they were most likely to participate in 
R-Tech during regular instructional time, at home using a home computer, at school, or during a free 
period. Staff on R-Tech campuses reported that many students experienced barriers to participating in 
R-Tech resulting from students’ general disinterest, involvement in extracurricular activities, and 
transportation limitations that made it difficult for students to stay after school or participate in morning 
programs. To address banters, districts made participation mandatory for some students and offered 
incentives, such as snacks, to encourage participation in R-Tech, and some districts adjusted bus 
schedules to enable students to stay after school for services. 

R-Tech students and school staff reported a range of benefits from participation in the program, including 
increased academic outcomes, greater confidence, and improved proficiency using technology resources. 
Students appreciated self-paced programs that focused on their particular - academic needs, and students 
liked the immediate and constructive feedback provided by some R-Tech software. Students and school 
staff noted that participation in R-Tech had increased students’ confidence, which led to greater 
participation in class activities and prompted some students to consider postsecondary educational 
opportunities. 

In addition to academic benefits, students enjoyed the convenience technology provides, such as carrying 
fewer books and typing instead of hand writing notes and essays. Students noted that access to online 
resources expanded their ability to gain information and facilitated research projects. Several students felt 
they benefited from increased proficiency using technology and that the skills they learned through 
R-Tech would assist them in college and in future employment. 
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Students’ dissatisfaction with R-Tech services generally resulted from school infrastructure issues rather 
than the program itself. For example, students described system challenges such as incompatibility 
between old computers and new software, slow Internet connections, and limited school bandwidth that 
prevented students from accessing the Internet at the same time. Several students also expressed 
complaints about districts’ software selections, noting R-Tech programs did not meet their instructional 
needs and provided insufficient instruction on key concepts. 
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Chapter 5 

R-Tech and Teachers 


Teachers in rural districts often experience reduced professional development opportunities because of 
their geographic isolation and the need to travel substantial distances in order to participate in out-of- 
district workshops or conferences. Technology has the potential to offset these limitations by providing 
access to online professional development and distance learning opportunities. Several states (e.g., 
Tennessee, Florida, Georgia, Kentucky) have implemented statewide programs designed to provide rural 
teachers with timely and relevant training through the use of technology-based training methods 
(Southern Regional Education Board [SREB], 2005). A 2004 survey of teachers in rural schools found 
that an increasing number of teachers used technology to access online professional development, meet 
recertification requirements, participate in professional enrichment activities, obtain advanced degrees, 
and learn new instructional strategies (SREB, p. 9). 

Recognizing the potential of technology to improve teacher performance in rural areas, R-Tech funding 
may be used to provide teacher professional development designed to guide the implementation of 
research- based instructional programs and support effective instruction in rigorous and diverse courses 
(TEA, 2008a, p. 4). In grant applications, all Cycle 1 grantee districts indicated that R-Tech programs 
would include professional development and training for teachers. This chapter presents findings about 
how R-Tech professional development was implemented in the pilot’s first year and how training may 
have affected teachers (Research Question 3). The chapter considers the following questions: 

• Flow do grantee districts and schools implement the teacher training component of the R-Tech 
program? 

• What types of training do teachers participate in as part of the R-Tech program? 

• What is the effect of R-Tech teacher training on teacher effectiveness? 

In addition, the chapter addresses principals’ expectations for R-Tech’ s effects on teachers and the 
opportunities beyond access to professional development that teachers may recognize from districts’ 
participation in the R-Tech grant. 

DATA SOURCES 

The chapter relies on data collected through fall 2008 and spring 2009 online surveys of teachers and 
principals in R-Tech districts, document analysis of district grant applications, as well as information 
gathered through site visit interviews with principals and focus group discussions with teachers. 
Additional information about the teacher and principal surveys, including administration procedures, 
response rates, respondent characteristics, supplemental data tables, and copies of respective surveys are 
included in Appendix A (principal survey) and Appendix B (teacher survey). More information on the 
districts identified for site visits and the types of R-Tech programs they implemented is included in 
Appendix D. 

As discussed in chapter 1 , larger proportions of teachers and principals responded to fall 2008 surveys 
than surveys administered in spring 2009. Flowever, information provided through fall surveys indicated 
that many R-Tech programs were not in place during the fall semester, and many survey respondents 
lacked basic information about the program. In light of this trend, this chapter’s discussion focuses more 
heavily on results from spring surveys because spring respondents had greater awareness of the R-Tech 
program and its professional development component. 
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PRINCIPALS’ EXPECTATIONS FOR R-TECH’S EFFECTS ON TEACHERS 


The fall 2008 and spring 2009 surveys of principals on R-Tech campuses contained two open-ended items 
addressing principals’ expectations for R-Tech’ s effects on teachers. The first question asked principals 
about their goals for R-Tech’ s effects on teachers and the second asked how they would know if goals had 
been met. Twenty principals entered responses to the open-ended items in fall 2008, and 40 entered 
responses for the spring 2009 survey. The sections that follow summarize principals’ responses. 

Fall 2008 

There were few common themes in principals’ responses to the fall survey. Six principals (30% of 
respondents) wrote that they hoped R-Tech would improve student-teacher interactions, and another six 
(30%) wrote that R-Tech would provide additional resources that would enhance classroom instruction. 
Two principals (10%) indicated that they wished teachers would become more aware of R-Tech, and two 
others expressed a desire to see teachers improve TAKS scores through the use of R-Tech resources. 
Principals wrote that they would know goals had been met if student achievement improved (65%) and 
through classroom observations of teachers (40%). 

Spring 2009 

Relative to principals’ written responses to the fall 2008, responses to the spring survey reflected an 
increased understanding of the R-Tech program and an increased focus on student outcomes. A third of 
principals responding in spring (33%) wrote that improving student outcomes was their central goal for 
teachers, and 20% wished teachers would become more aware of R-Tech. Several principals responded 
that they expected to see teachers increase their use of technology in the classroom (15%). One principal 
wrote, “I want our teachers to receive training and utilize technology equipment in class. I want integrated 
lessons and technology almost daily as part of the lesson.” 

Principals’ responses to the open-ended item asking how they would know goals had been met also 
suggest that principals had increased their awareness of the R-Tech program in spring 2009. While no 
principals responding to the fall survey referenced using teacher participation data as a source of 
information, 20% of principals responding in the spring cited documentation of teacher software use and 
participation in training as a means to measure whether goals had been met. 1 Principals also cited 
improved student achievement outcomes (55%) and classroom observations (18%) as indicators that goals 
had been achieved. 


'As discussed in chapter 1, R-Tech districts are required to collect information about student and teacher 
participation in R-Tech services through a system of data uploads provided to TEA for summer 2008, fall 2008, and 
spring 2009. 
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Teacher Awareness of the R-Tech Program and Its Goals 

Increasing teachers’ awareness of the R-Tech program was a common theme in principals’ goals for 
teachers, as indicated in their responses to the fall and spring surveys. The R-Tech teacher surveys (fall 
2008 and spring 2009) included items that gauged teachers’ awareness of the R-Tech program and its 
goals, and results indicate that few teachers were aware of R-Tech during the pilot’s first year. Both 
surveys included a set of statements about R-Tech, and asked teachers to indicate their level of agreement 
with each statement or whether they didn ’t know how to respond to the statement. Researchers coded their 
responses: strongly disagreed (-10), disagreed (-5), unsure (0), agreed (5), and strongly agreed (10) as a 
means to clearly illustrate variations in teachers’ levels of agreement. Table 5.1 presents teachers’ mean 
responses, sorted in terms of the “Spring 2009” column. Values closer to 10 indicate higher levels of 
agreement and values closer to -10 indicate higher levels of disagreement. 


Table 5.1 

Teachers’ Levels of Agreement: R-Tech Goals and Outcomes, as a Mean of Respondents: Fall 
2008 and Spring 2009 


Statements about Goals 

Fall 2008 
(N=l,213) 

Spring 2009 
(N=568) 

Overall, I am pleased with the services provided by R-Tech. 

2.1 

2.8 

R-Tech is positively affecting student achievement on campus. 

2.1 

2.8 

Vender services are aligned with the TEKS/TAKS. 

2.1 

2.6 

Vender services are aligned with our campus goals. 

1.9 

2.2 

Goals are clearly communicated. 

1.0 

1.4 

I use information from student’s Personal Education Plans when I 
plan classroom instruction. 

0.1 

0.2 


Sources: R-Tech Teacher Survey, fall 2008, spring 2009. 

Note. Items have been coded: strongly disagree (-10), disagree (-5), unsure (0), agree (5), strongly agree (10). 
Items closer to 10 indicate higher levels of agreement with each statement. 


Across all statements, teachers’ average levels of agreement increased from fall to spring. In addition the 
percentages of teachers responding that they didn ’t know to each item decreased from fall to spring (see 
Table B.5 in Appendix B). This pattern suggests that teacher awareness of R-Tech improved somewhat 
over the course of first year implementation. Although many teachers were still unfamiliar with R-Tech in 
spring 2009, spring survey responses indicate that they were pleased with R-Tech services (2.8) and that 
the program was positively affecting student achievement (2.8). Statements receiving the lowest ratings 
addressed the clear - communication of R-Tech goals (1.4) and teachers’ use of PEPs to plan instruction 
(0.2). These findings underscore principals’ concerns about teacher awareness of R-Tech and raise 
questions as to whether the program had a meaningful effect on teachers in its first year. 
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R-TECH PROFESSIONAL DEVELOPMENT 


In grant applications, all R-Tech districts indicated plans to offer vendor-provided training designed to 
introduce teachers to software purchased with R-Tech funds, and many districts planned to offer 
additional professional development activities in support of R-Tech. Table 5.2 presents the additional 
training areas that were common across at least five district applications, and indicates that the largest 
proportions of districts planned additional training in vertical alignment (28%), instructional technology 
(27%), and pedagogical best practices (23%). 


Table 5.2 

R-Tech Professional Development Opportunities for Teachers, District Grant Applications: 2008 


Professional Development Topic 

Number of 
Districts 

Percentage of 
Districts 
(N=63) 

Vertical alignment, collaboration, mentoring 

18 

28.1% 

Instructional technology 

17 

26.5% 

Best practices/pedagogy 

15 

23.4% 

Distance learning 

11 

17.4% 

Training in computer hardware 

11 

17.0% 

R-Tech pilot program 

9 

14.0% 

TEKS/TAKS preparation 

9 

14.0% 

Educating at-risk student groups 

7 

10.9% 

Dual credit opportunities 

6 

9.4% 

Multimedia training 

5 

7.8% 


Source: Analysis of R-Tech district applications. 

Note. Percentages will not total to 100%. Districts indicated multiple training topics. 


The fall 2008 and spring 2009 surveys asked teachers whether they participated in training related to 
R-Tech, the number of hours spent in R-Tech training, and the total number of hours spent in any type of 
professional development activity since May 2008 (i.e., across R-Tech’s first implementation year). 
However, fewer than 5% of respondents to the fall survey were aware of training related to R-Tech. 

Given the small percentage of fall survey respondents with knowledge of R-Tech professional 
development activities, the following discussion is limited to results from the spring survey 
administration when teachers had greater awareness of R-Tech training opportunities. 

In spring 2009, teachers indicated that had participated in an average of 21 hours of professional 
development, and 38% of teachers (215 individuals) knew that they had participated in R-Tech training. 
These teachers estimated they spent 13 hours, on average, in professional development activities related 
to R-Tech. However, not all teachers were clear that they received R-Tech training. Thirty-seven percent 
of respondents reported that they did not know if they had participated in R-Tech training over the course 
of the program’s first year. In response to an open-ended item asking about the most useful aspects of 
R-Tech training, one teacher explained, “I don’t know, exactly, what all [of the professional development] 
was considered R-Tech.” 
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Types of R-Tech Professional Development Offered 

Spring 2009 survey respondents who knew they participated in R-Tech professional development 
indicated the content of the training they received through the grant. Figure 5. 1 presents the percentage of 
teachers who responded to each type of professional development. Most respondents reported they 
participated in training related to TEKS/TAKS preparation (83%), integrating technology into instruction 
(77%), working with at-risk students (76%), using instructional hardware (67%), using vendor software 
(66%), and curricular alignment (60%). 


TEKS/TAKS preparation 
Technology based instruction 
Working with at-risk students 
Using instructional hardware 
Introduction to vendor software 
Curricular alignment 
College readiness preparation 
Study groups 
Visit another district's R-Tech program 
Course for college credit 



Percentage of Respondents 
(N=215) 


Figure 5.1. Content of R-Tech professional development: spring 2009. 

Source: R-Tech Teacher Survey, spring 2009. 

Notes. Percentages will not total to 100. Teachers indicated multiple topics for R-Tech professional development. 
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Format of Training 

Teachers who participated in R-Tech training also described the format of professional development 
opportunities, indicating whether the trainings were conducted online , face-to-face, or presented using 
other formats. As presented in Figure 5.2, across all training topics, a majority of teachers participated in 
face-to-face training. With the exception of “College readiness preparation” (21%), fewer than 20% of 
teachers reported participating in online professional development. 


Study groups 
Introduction to vendor software 
Technology-based instruction 
Curricular alignment 
Working with at-risk students 
Using instructional hardware 
TEKS/TAKS preparation 
Course for college credit 
College readiness preparation 



0% 


20% 40% 60% 

Percentage of Respondents 


80% 


100% 


■ Face-to-Face DOnline □ Other 


Figure 5.2. Format of R-Tech professional development, as a percentage of respondents: spring 
2009. 

Source: R-Tech Teacher Survey, spring 2009. 

Note. The number of respondents to each item ranges from 17 to 179 because only teachers indicating they received 
each type of training selected the format of the training. No space was provided for teachers to indicate what “other” 
formats may have been. 


Online training. Although few surveyed teachers participated in online professional development 
activities provided through R-Tech, district- provided data reported to TEA for the grant’s first 
implementation year indicates that more than 800 teachers (about 22% of all teachers on R-Tech 
campuses) received R-Tech funded online training during the 2008-09 school year. Similar to the student 
use data districts report to TEA, districts also report information about teachers’ use of online training 
resources for the summer 2008, fall 2008, and spring 2009 reporting periods. Table 5.3 presents the 
average number of hours teachers used online resources for the fall 2008 and spring 2009 reporting 
periods. District reports indicated that no teachers accessed R-Tech online professional development 
resources in summer 2008. 
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Table 5.3 

Average Hours Teachers Spent in R-Tech Online Professional Development Activities: 
Fall 2008 and Spring 2009 


Reporting Period 

Elementary 

School 

Teachers 3 

(n=28) 

Middle 

School 

Teachers 

(n=281) 

High School 
Teachers 
(n=449) 

Combined 
Middle and 
High School 
Teachers 
(n=44) 

All Teachers 
(N=802) 

Fall 2008 

8.3 

10.2 

9.6 

4.7 

9.5 

Spring 2009 

1.5 

8.7 

6.0 

7.2 

6.9 

Year Total 

9.7 

18.8 

15.6 

11.9 

16.3 


Source: Texas Education Agency (TEA) R-Tech Teacher Upload data, fall 2008, spring 2009. 
“Seven R-Tech campuses are elementary schools that enroll students in Grade 6. 


Results indicate that most teachers spent more time, on average, with R-Tech’ s online professional 
development resources in the fall than in the spring (10 hours vs. 7 hours for “All Teachers”), and middle 
school teachers spent the most time using R-Tech (19 hours). Notably, the average hours elementary 
school teachers spent using R-Tech online professional development activities dropped from 8 hours in 
fall 2008 to about 2 hours in spring 2009. The reduced use may suggest that online training activities were 
not well matched to elementary school teachers’ needs. 

Teachers’ Perceptions of R-Tech Professional Development 

The spring 2009 teacher survey also contained open-ended items asking teachers about the most and least 
useful aspects of the training they received through R-Tech. In addition, teachers participating in focus 
group discussions conducted as part of site visits to R-Tech districts discussed their perceptions of R-Tech 
training. The following sections summarize teachers’ responses. 

The most useful aspects of R-Tech professional development. Of the 215 teachers who knew they 
participated in R-Tech training across the pilot’s first year, 68% (147 teachers) entered written responses 
describing the aspects of the training that they felt were most useful. Researchers coded written entries to 
identify common themes across responses and calculated the percentage of responses by theme. Table 5.4 
presents the percentages of teachers indicating useful aspects of training. The largest proportion of 
teachers (29%) indicated that learning how to integrate technology into instruction was the most useful 
aspect of training. Teachers noted the value of learning to integrate technology resources such as laptops, 
digital whiteboards (i.e., Smart Boards), and document cameras into classroom instruction. A somewhat 
smaller proportion of teachers (26%) wrote that the introductions to vendor programs and software were 
valuable. Teachers appreciated time spent with product vendors and learning software capabilities. Other 
teachers were pleased to learn how to use new technology resources (17%) and to have the opportunity 
for hands-on practice with R-Tech resources (12%). 


75 




Table 5.4 

Teachers’ Perceptions of the Most Useful Aspects of R-Tech Professional 
Development, as a Percentage of Respondents: Spring 2009 


Most Useful Aspect 

All Respondents 
(N=147) 

Learning how to integrate technology into instruction 

28.6% 

Becoming acquainted with the program/software 

25.9% 

Learning to use new hardware 

17.0% 

Hands-on practice 

12.2% 

Recognizing the benefits for students 

11.6% 

Face-to-face assistance with trainers and other teachers 

10.2% 

How to use reports and data to monitor students’ progress 

4.8% 

How to relate programs to TAKS 

4.1% 

Learning about the R-Tech pilot 

3.4% 


Source : R-Tech Teacher Survey, spring 2009. 

Notes. The number of respondents (N) represents teachers responding to an open-ended item. 

Percentages will not total to 100 because some teachers’ responses included information 

counted in multiple categories. 

During focus group discussions conducted as part of site visits, teachers provided more information about 
what they found useful about training. At one high school implementing a technology immersion project 
in which all teachers and students received laptops, teachers were enthusiastic about the potential for 
laptops to transform teaching. One focus group teacher explained, “I was amazed... [laptops are] so much 
different than our desktops.” Another teacher added “I think the whole thing [R-Tech] is overwhelming 
because [there’s] just so much you can do [using laptops].” A middle school teacher on a campus 
implementing a self-paced tutoring program appreciated learning how to use the software to differentiate 
instruction. “The training [addressed] how to give individualized assignments... so you know they’re 
[students] doing what you want them to do.” Focus group teachers also said that interactive, hands-on 
training increased their retention of newly learned skills. 

The least useful aspects of R-Tech professional development. Only 31% of teachers (67 respondents) 
who participated in R-Tech professional development entered written comments addressing the least 
useful aspect of training. Similar to findings for the most useful aspects of training, researchers coded 
teachers’ responses across common themes, and calculated the percentage of responses by theme. As 
presented in Table 5.5, the largest proportion of teachers objected to training that was not relevant to the 
courses they taught (27%). For example, one science teacher found the training “worthless” because the 
campus’ R-Tech vendor provides remediation in reading comprehension. The teacher indicated the 
science team is “not going to use the [the program] ... because [they] can’t use [the program].” Another 
teacher noted that unless access to computers was expanded, it would be difficult to incorporate training 
into classroom instruction. “I could not see how to use the material in my classroom,” wrote the teacher, 
“unless I had an opportunity to use computers much more often than they are available.” 

Nineteen percent of teachers planned to use R-Tech resources but reported that the training did not 
provide enough in-depth information to implement resources effectively. One teacher wrote, “[The 
training] only went over the basics.” In contrast, 12% of teachers felt overwhelmed because training 
addressed “too much information in a short period of time.” One survey respondent wrote, “The training 
was very compressed. I would have learned more from a longer in-service.” Several survey respondents 
expressed an interest in continuing training. One teacher reported it would be beneficial for teachers “to 
get a refresher course later on.” Another teacher agreed additional training would be valuable, writing “I 
need more professional development.” 
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Table 5.5 

Teachers’ Perceptions of the Least Useful Aspects of R-Tech 
Professional Development, as a Percentage of Respondents: Spring 2009 


Least Useful Aspect 

All Respondents 
(N=67) 

Content not relevant to field of instruction 

26.9% 

Not enough in-depth information 

19.4% 

Too much material in a short period of time 

11.9% 

Program glitches 

7.4% 

Too early in the year-need refreshers 

6.0% 

Already familiar with the program 

6.0% 

Hardware problems hindered practicing with program 

4.5% 

Not enough practice/hands-on opportunities 

4.5% 


Source : R-Tech Teacher Survey, spring 2009. 

Notes. The number of respondents (N) represents teachers responding to an open- 
ended item. Percentages will not total to 100 because some teachers’ responses 
included information counted in multiple categories. 

Teachers participating in site visit focus groups expressed similar concerns about training. In a district 
that implemented a self-paced tutorial program for reading/ELA, math and science teachers felt they did 
not need to participate in training in software that did not support their subject areas. Teachers in a 
technology immersion program felt that they should have been provided with laptops and training prior to 
students receiving laptops. “I would much rather have the teachers get their computers and use them for a 
year and then give them to the kids,” explained one teacher. A principal in the immersion district felt that 
the vendor-provided training in the use of resources was ineffective. “The [vendor] training, in my 
opinion, did no real service for them [teachers],” explained the principal, “I would have much rather had 
them have more time for content area integration [training].” 

ADDITIONAL OPPORTUNITES PROVIDED BY R-TECH 

During spring 2009 site visits to R-Tech districts, researchers asked teachers participating in focus groups 
about the additional opportunities the grant may have created for teachers. Responses indicate that 
R-Tech increased teachers’ ability to access technology, diversify and individualize instruction, and 
increase student engagement in learning. In some districts, however, principals and teachers said that 
teachers had not made full use of the opportunities provided by R-Tech during the program’s first year. 

Increased Access to Technology 

In focus group discussions, many teachers cited increased access to technology as a primary benefit to 
R-Tech participation. Teachers in a district that used R-Tech funding to add a computer lab that was open 
and staffed throughout the school day and before and after school said that they were now able to assign 
homework that required students to use technology. “It's nice to be able to actually assign something with 
technology and know it’s not going to take a week,” explained a teacher. “They [students] can’t use ‘not 
having a computer’ as an excuse.” Another teacher in the focus group appreciated that the district had 
hired a full time facilitator for the lab, noting “the biggest thing is we know someone is in there [the lab], 
so we can send our kids before, during, and after school.” Teachers in a district implementing a 
technology immersion program said that they used laptops and Internet-based activities in their 
classrooms, but assigning homework was difficult because many students did not have Internet access at 
home. 
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Increased Ability to Differentiate Instruction 

Focus group teachers said that increased access to technology enabled differentiated and individualized 
instruction. Teachers in one district sent groups of students to computer labs to access R-Tech resources 
while other students remained in the classroom and received direct instruction. Teachers in another 
district explained: 

I like the aspect of the individuality of it. There is no way to teach one-on-one in the classroom. 
[But with the program], students can work on the areas that they need and the student sitting next 
to them can be working on the area that he/she needs. . . and it’s not a big strain on us to do that. 

Teachers in several districts said that the detailed reports provided by R-Tech software programs enabled 
greater differentiation in classroom instruction. One such teacher explained that the reports provided a 
“different perspective of students’ successes and struggles” relative to more traditional forms of 
assessment. 

Increased Student Engagement 

Across site visit districts, teachers said that students were more engaged when instruction included 
technology. One teacher explained, “[Technology] makes it [instruction] more interesting when they’re 
[students] hands on and using computers, which they love.” In another district, teachers reported, “Our 
kids seem to really get excited about technology and just being on the computer and doing something 
different than a textbook.” Another teacher added that students were engaged by the instructional games 
included in R-Tech software, which made instruction easier. “You’re not fighting them [when they use 
the software],” explained the teacher. 

Missed Opportunities 

Although many teachers said that R-Tech increased opportunities, some interview respondents felt that 
R-Tech resources were underused in the program’s first year. One principal noted that increased access to 
technology only provides opportunities and teachers must take advantage of opportunities to realize 
benefits. The principal explained, “I would say [R-Tech] has created opportunities. I don’t know if 
anything is being done with [the opportunity], but the opportunity is there.” A principal on another 
campus expressed a similar view: 

I don’t think we’ve done a good job of making the whole staff aware of the possibilities. . . .1 don’t 
think [teachers] see that it [R-Tech] could be a really powerful component.. . .1 wished we used it 
more effectively [during the first year] — it’ s a work in progress. 

A teacher focus group participant agreed, noting “As far as opportunities go, some [teachers] are taking 
greater advantage than others.” A principal in another district planned to encourage greater teacher 
participation in the grant’s second year, explaining “With increased technology comes an increased 
expectation [that it will be used].” 

R-TECH’S FIRST YEAR EFFECTS ON TEACHERS 

The fall 2008 and spring 2009 teacher surveys contained items that asked teachers about how R-Tech may 
be affecting their professional growth, knowledge, and instruction. Teachers were provided with a list of 
statements about R-Tech’ s effects and were asked to indicate their level of agreement with each statement 
or whether they didn ’t know if they had experienced the effect. Researchers coded their responses: 
strongly disagreed (-10), disagreed (-5), unsure (0), agreed (5), and strongly agreed (10) as a means to 
illustrate variations in teachers’ levels of agreement. Table 5.6 presents teachers’ mean responses for the 
fall and spring survey administrations. Values closer to 10 indicate higher levels of agreement and values 
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closer to -10 indicate higher levels of disagreement. Results again indicate that teachers gained greater 
awareness of R-Tech over the pilot’s first year. Across all statements, teachers’ average levels of 
agreement increased from fall to spring. In addition, the percentage of teachers who responded that they 
didn’t know to each statement decreased across survey administrations (see Table B.6 in Appendix B). In 
spring, most survey respondents expressed a greater awareness of technology-based learning (3.3), 
increased opportunities for technology-based professional development (2.8), and improved technical 
skills (2.7). However, smaller percentages of teachers agreed that growth has translated into improved 
instruction (2.2) or improved lesson plans (1.8). 


Table 5.6 

Effects of R-Tech Implementation on Teachers, as a Mean of Respondents, by Semester: Fall 
2008 and Spring 2009 


Positive Effects 

Fall 2008 
(N=l,213) 

Spring 2009 
(N=568) 

I have a greater awareness of technology-based learning opportunities 
for students. 

2.6 

3.3 

I have the opportunity to participate in technology-based professional 
development. 

2.6 

2.8 

My technical skills and abilities have improved. 

2.1 

2.7 

I have a better understanding of the needs of at-risk students. 

1.8 

2.3 

My teaching has improved. 

1.7 

2.2 

My lesson plans have improved. 

1.4 

1.8 


Source: R-Tech Teacher Survey, fall 2008, spring 2009. 

Note. Items have been coded: strongly disagree (-10), disagree (-5), unsure (0), agree (5), strongly agree (10). 
Items closer to 10 indicate higher levels of agreement with each statement. 


In site visit interviews, several principals provided explanations for R-Tech’ s relatively weak effects on 
teachers over its first year. One principal said, “When you first get a program it’s hard to have that buy-in 
with teachers, and it does take some time. . .and proven success with the kids.” A principal in another 
district felt that weak communication about R-Tech had affected teacher outcomes. “It [communication] 
could have been better. I can tell you that,” said the principal. “I think those [teachers] that use it [R-Tech 
resources] on a regular basis know more about it, but there are some that probably don’t know what it is.” 

SUMMARY 

Results presented in this chapter indicate that while principals expect R-Tech to affect teachers in ways 
that lead to improved student outcomes, R-Tech has had a relatively weak effect on teachers’ instruction 
during its first implementation year. Few surveyed teachers were aware of R-Tech in fall 2008, and while 
awareness of the program improved from fall to spring, less than half of teachers responding to the spring 
2009 survey indicated that R-Tech had improved teaching or lesson plans, and less than a third used 
students’ PEPs to guide instruction. Teachers were more likely to report R-Tech’ s effects in terms of 
access to technology resources, training in the use of resources, and greater awareness of opportunities for 
students to participate in technology-based learning. 

Although all Cycle 1 grantee districts indicated plans to offer professional development opportunities as 
part of R-Tech, relatively few surveyed teachers had participated in training related to R-Tech during the 
pilot’s first year. Teachers who received R-Tech training appreciated learning how to integrate technology 
into classroom instruction and learning about vendor-provided resources. Most training provided through 
R-Tech was delivered in face-to-face formats, and few teachers reported participating in online 
professional development activities. District-provided data on teachers’ access to online training indicate 
that middle school teachers spent about 19 hours, on average, using online resources relative to 16 hours 
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for high school teachers, but the amount of time high school and middle school teachers spent using 
resources decreased from fall to spring. 

To some extent, R-Tech’s weak effects on teachers during its first year of implementation may have 
resulted from poor communication of grant goals and opportunities. Surveyed teachers indicated that 
R-Tech’s goals had not been clearly communicated, and interviewed principals suggested that lack of 
communication about the pilot limited teacher awareness. Principals also explained that more time was 
needed to establish R-Tech programs and to encourage teacher buy-in. 


80 



Chapter 6 

R-Tech Student Outcomes 


A central focus of the evaluation is whether R-Tech services affect students’ academic outcomes, 
including TAKS scores (Research Question 4). However, as discussed in chapter 1, standardized tests 
may lack the sensitivity to detect the effects of supplemental programs in which students spend a 
relatively small proportion of their instructional time. Further, it is difficult for researchers to isolate the 
effects of supplemental instruction from the many other factors that affect students’ performance on 
standardized tests. To offset the limitations of standardized tests, the evaluation considers a variety of 
student outcomes, including graduation outcomes, course completion rates, and indicators of college 
readiness. While the broader range of outcomes will be included in the final evaluation report (fall 2010), 
the only 2009 outcome data available at the time of the second interim report’s writing were students’ 
spring 2009 TAKS scores. Therefore, the analyses presented in this chapter are limited to students’ testing 
outcomes and consider the following questions: 

• What is the effect of R-Tech on students’ TAKS outcomes? 

• Is there a relationship between students’ access times and TAKS outcomes? 

• Is there a relationship between R-Tech program configurations and students’ TAKS outcomes? 

The chapter begins with a discussion of data sources included in analyses and an overview Texas’ system 
of standardized testing. Subsequent sections present analyses of students’ progress toward meeting TAKS 
standards and the results of HLM analyses of the effects of access time and program type on students’ 
TAKS outcomes. 

DATA SOURCES 

The chapter incorporates a range of quantitative data sources. Analyses draw on students’ testing 
outcomes and demographic data included in PEIMS, as well as information included in district-provided 
student usage data. As discussed in chapter 1, R-Tech districts submitted information on students’ 
program participation for the summer 2008, fall 2008, and spring 2009 reporting periods through a system 
of data uploads hosted by TEA. District reports included the average number of hours per week individual 
students accessed R-Tech resources, as well as the subjects they studied. 

In addition, analyses consider the program configurations implemented during R-Tech’ s first year. As 
discussed in chapter 3, researchers used R-Tech grant applications and progress reports to identify five 
non-discrete categories of R-Tech programs: (1) self-paced instructional programs, (2) dual 
credit/distance learning programs, (3) one-to-one tutoring with online instructional support, (4) 
school-wide technology immersion programs, and (5) iPods loaded with instructional content. The 
categories are non-discrete because districts may implement R-Tech differently at the high school and 
middle school levels. For example, a district may choose to implement self-paced tutoring for its middle 
school students but offer dual credit coursework for high school students. Researchers further identified 
whether R-Tech was implemented as a supplemental program in which instruction was provided outside 
of students’ regularly scheduled classes or whether R-Tech services were incorporated into regular 
classroom instruction (i.e., non-supplemental programs). 
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OVERVIEW OF THE TEXAS ASSESSMENT OF KNOWLEDGE AND SKILLS (TAKS) 


The TAKS is Texas’ criterion-referenced assessment that measures students’ mastery of the state’s 
content standards, the TEKS. At the middle school level, TAKS assesses reading and mathematics at 
Grades 6, 7, and 8, writing at Grade 7, and science and social studies at Grade 8. At the high school level, 
TAKS assesses reading/ELA and mathematics at Grades 9, 10, and 11, and science and social studies at 
Grades 10 and 11. This evaluation incorporates several types of TAKS scores. 

• Met the standard. This score represents satisfactory academic achievement. Students who meet 
this standard performed at a level that was at or somewhat above the state passing standard. Thus, 
students demonstrated a sufficient understanding of the knowledge and skills measured at the 
grade level. 

• Commended performance. This score represents high academic achievement. Students who 
meet this standard performed at a level that was considerably above the state passing standard. 
Therefore, students demonstrated a thorough understanding of the knowledge and skills measured 
at the grade level. 

• TAKS scale score. The scale score is a statistic that provides a comparison of scores with a 
standard set at 2100 for each grade level. The scale score can be used to determine whether a 
student met the minimum standard (a scale score of 2100) or achieved commended performance 
(a scale score of 2400 or above), but it cannot be used to evaluate a student’s progress across 
grades or subject areas. TAKS scale scores are used to calculate standardized scores for this 
evaluation. 

Texas has phased in increasingly rigorous passing standards on the TAKS. In 2004-05, passing standards 
recommended for reading, mathematics, writing, social studies, and Grade 5 science by the State Board of 
Education panel were fully implemented. For the newer Grade 8 science test, the panel-recommended 
standard had to be met in 2007-08. For this evaluation, all TAKS scores reported are based on 
panel-recommended standards. 

PROGRESS IN MEETING TAKS STANDARDS 

TAKS Reading 

One measure of student academic outcomes is their progress toward meeting TAKS passing and 
commended performance standards. The data in Table 6.1 represent two cohorts of students in R-Tech 
schools — those who participate in R-Tech services and those who do not (i.e., non-participants) — who 
took the TAKS reading test in both spring 2008 and spring 2009. 1 Using these data, one can compare the 
absolute TAKS reading performance of students who participated in R-Tech activities and students who 
did not participate in R-Tech activities. 


'A cohort is a group of students who experience certain events in a specified period of time. For example, the cohort 
of participating students attended R-Tech campuses in 2008-09, took part in R-Tech instructional activities at those 
campuses, and took the TAKS reading test in both spring 2008 and spring 2009. The cohort of non-participating 
students attended R-Tech campus in 2008-09, did not take part in R-Tech instructional activities, and took the TAKS 
reading test in both spring 2008 and spring 2009. 
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Table 6.1 

Cohort Longitudinal TAKS Passing and Commended Performance Rates for Reading: 
R-Tech Participants and Non- Participants 


TAKS Test 

Group 

N 

2007-08 

Percent 

2008-09 

Percent 

2008 to 2009 
Percentage 
Change 

Met Standard 

Grade 5 to 6 

Participants 

403 

82.9 

91.8 

8.9 


Non- Participants 

683 

85.8 

94.1 

8.3 

Grade 6 to 7 

Participants 

1,505 

90.4 

82.5 

-7.9 


Non- Participants 

2,117 

94.6 

90.4 

-4.2 

Grade 7 to 8 

Participants 

1,537 

77.4 

90.5 

13.1 


Non- Participants 

3,020 

93.7 

96.6 

2.9 

Grade 8 to 9 

Participants 

1,715 

91.4 

90.9 

-0.5 


Non- Participants 

2,745 

94.1 

93.5 

-0.6 

Grade 9 to 10 

Participants 

1,723 

89.7 

88.2 

-1.5 


Non- Participants 

3,566 

93.0 

91.8 

-1.2 

Grade 10 to 11 

Participants 

1,475 

91.2 

93.3 

2.1 


Non- Participants 

3,360 

91.9 

95.4 

3.5 

Total 

Participants 

8,358 

87.9 

89.2 

1.3 


Non- Participants 

15,491 

93.0 

93.7 

0.7 

Commended Performance 

Grade 5 to 6 

Participants 

403 

27.5 

33.5 

6.0 


Non- Participants 

683 

29.4 

50.5 

21.1 

Grade 6 to 7 

Participants 

1,505 

38.9 

22.5 

-16.4 


Non- Participants 

2,117 

51.5 

33.4 

-18.1 

Grade 7 to 8 

Participants 

1,537 

20.9 

36.4 

15.5 


Non- Participants 

3,020 

35.6 

54.4 

18.8 

Grade 8 to 9 

Participants 

1,715 

42.0 

18.3 

-23.7 


Non- Participants 

2,745 

54.2 

23.5 

-30.7 

Grade 9 to 10 

Participants 

1,723 

29.8 

12.8 

-17.0 


Non- Participants 

3,566 

37.2 

19.0 

-18.2 

Grade 10 to 11 

Participants 

1,475 

16.7 

26.5 

9.8 


Non- Participants 

3,360 

17.4 

29.6 

12.2 

Total 

Participants 

8,358 

29.9 

23.4 

-6.5 


Non- Participants 

15,491 

37.2 

32.4 

-4.8 


Sources: Texas Education Agency (TEA) R-Tech Student Upload data: fall 2008 and spring 2009. Public 
Education Information Management System (PEIMS) student demographic data from fall 2008. Texas 
Assessment of Knowledge and Skills 2008 and 2009 individual student data from TEA. 

Notes. Students attended a campus participating in the R-Tech program in 2008-09. Participants took part 
in R-Tech instructional activities at one of the participating campuses. Non-participants attended an R-Tech 
participating campus but did not take part in R-Tech instructional activities. The grade level is the student’s 
grade in 2008-09. 
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Results show that students participating in R-Tech had lower TAKS passing rates in both spring 2008 and 
spring 2009 than students who were not participating in R-Tech. The overall passing rate gain of 1.3 
percentage points for participants was slightly larger than the overall gain of 0.7 percentage points for 
non-participants. Participants had larger gains (or smaller losses) than non-participants at Grades 6, 8. and 
9, but smaller gains (or larger losses) at Grades 7, 10, and 11. The largest gain by participants was 13.1 
percentage points at Grade 8. R-Tech participants also had lower commended performance rates than 
non-participants in both spring 2008 and spring 2009. The overall commended performance rate changes 
favored non-participants (a loss of -6.5 percentage points for participants and a loss of -4.8 percentage 
points for non-participants). Participants had smaller commended performance losses than 
non-participants at Grades 7, 9, and 10, but smaller gains at Grades 6, 8, and 11. 

TAKS Mathematics 

Table 6.2 compares the TAKS mathematics performance of participating and non-participating students 
in R-Tech schools. Similar - to TAKS reading, these data represent cohorts of students who have taken the 
TAKS mathematics test in both spring 2008 and spring 2009. The table shows that students participating 
in R-Tech had considerably lower mathematics passing rates in both spring 2008 and spring 2009 than 
non-participants. However, passing rate gains favored participating students. The overall gains were 1.8 
percentage points for participants and 0.2 percentage points for non-participants. Participants had larger 
gains (or smaller losses) than non-participants at Grades 7, 8, 10, and 11, but larger losses at Grades 6 and 
9. 


84 



Overall, students had greater difficulty meeting commended performance standards for mathematics 
compared to reading. As with TAKS reading, participants had lower commended performance scores than 
non-participants. The overall commended performance changes favored non-participating students. The 
overall changes were -1.4 percentage points for participants and -0.1 percentage points for 
non-participants. At Grades 7 and 10, participants had smaller losses, while at Grades 6, 8, 9, and 11, 
non-participants had larger gains. 


Table 6.2 

Cohort Longitudinal TAKS Passing and Commended Performance Rates for Mathematics: 
R-Tech Participants and Non- Participants 


TAKS Test 

Group 

N 

2008 

Percent 

2009 

Percent 

2008 to 2009 
Percentage 
Change 

Met Standard 

Grade 5 to 6 

Participants 

400 

83.8 

78.3 

-5.5 


Non-Participants 

684 

85.1 

83.0 

-2.1 

Grade 6 to 7 

Participants 

1,506 

77.4 

75.6 

-1.8 


Non-Participants 

2,122 

86.4 

82.9 

-3.5 

Grade 7 to 8 

Participants 

1,539 

66.7 

71.7 

5.0 


Non-Participants 

2,990 

87.0 

87.0 

0.0 

Grade 8 to 9 

Participants 

1,685 

70.3 

63.3 

-7.0 


Non-Participants 

2,724 

80.5 

75.5 

-5.0 

Grade 9 to 10 

Participants 

1,723 

58.0 

57.5 

-0.5 


Non-Participants 

3,539 

74.1 

68.3 

-5.8 

Grade 10 to 11 

Participants 

1,454 

62.2 

79.5 

17.3 


Non-Participants 

3,326 

71.0 

84.5 

13.5 

Total 

Participants 

8,307 

67.6 

69.4 

1.8 


Non-Participants 

15,385 

79.2 

79.4 

0.2 

Commended Performance 

Grade 5 to 6 

Participants 

400 

32.0 

22.0 

-10.0 


Non-Participants 

684 

34.4 

40.4 

6.0 

Grade 6 to 7 

Participants 

1,506 

27.0 

11.4 

-15.6 


Non-Participants 

2,122 

40.6 

22.7 

-17.9 

Grade 7 to 8 

Participants 

1,539 

11.4 

14.7 

3.3 


Non-Participants 

2,990 

20.0 

25.8 

5.8 

Grade 8 to 9 

Participants 

1,685 

13.2 

17.4 

4.2 


Non-Participants 

2,724 

19.6 

25.0 

5.4 

Grade 9 to 10 

Participants 

1,723 

14.3 

8.5 

-5.8 


Non-Participants 

3,539 

23.7 

14.2 

-9.5 

Grade 10 to 11 

Participants 

1,454 

13.5 

22.8 

9.3 


Non-Participants 

3,326 

15.9 

26.4 

10.5 

Total 

Participants 

8,307 

16.5 

15.1 

-1.4 


Non-Participants 

15,385 

23.4 

23.3 

-0.1 


Sources: Texas Education Agency (TEA) R-Tech Student Upload data: fall 2008 and spring 2009. Public 
Education Information Management System (PEIMS) student demographic data from fall 2008. Texas 
Assessment of Knowledge and Skills 2008 and 2009 individual student data from TEA. 

Notes. Students attended a campus participating in the R-Tech program in 2008-09. Participants took part in 
R-Tech instructional activities at one of the participating campuses. Non-participants attended an R-Tech 
participating campus but did not take part in R-Tech instructional activities. The grade level is the student’s grade 
in 2008-09. 
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TAKS Social Studies, Science, and Writing 

While the TAKS reading and mathematics tests are administered annually, TAKS science, social studies, 
and writing tests are administered periodically. Because of this, cross-sectional, rather than cohort data, 
are presented in the tables that follow. Instead of reporting TAKS scores for the same group of students at 
two points in time (spring 2008 and spring 2009), researchers reported TAKS scores for two different 
groups of students at two points in time. For example, in Table 6.3, science scores are reported for 
students who were in Grade 8 in spring 2008 (baseline measure) and for students who were in Grade 8 in 
spring 2009. For both years, scores are reported for students who participated and did not participate in 
R-Tech in 2008-09. 

TAKS Science. In Table 6.3, the results for TAKS science show that, similar to reading and mathematics, 
students participating in R-Tech had lower passing rates and commended performance rates than 
non-participants. The 2008 to 2009 change in passing rates was similar for participant groups and 
non-participant groups (0.6 percentage points for participant groups and 0.2 percentage points for 
non-participant groups). The 2008 to 2009 change in commended performance rates was slightly higher 
for the non-participant groups (2.5 percentage points for non-participant groups and 0.4 percentage points 
for participant groups). 


Table 6.3 

Cross-Sectional Longitudinal TAKS Passing and Commended Performance Rates for Science: 
R-Tech Participants and Non- Participants 




2008 

2009 

2008 to 2009 

TAKS Test 

Group 

N 

Percent 

N 

Percent 

Difference 

Met Standan 

i 

Grade 8 

Participants 

1,717 

59.6 

1,923 

61.8 

2.2 


Non- Participants 

3,427 

74.7 

3,617 

77.0 

2.2 

Grade 10 

Participants 

1,640 

62.6 

2,081 

58.0 

-4.6 


Non- Participants 

4,110 

68.3 

4,260 

68.1 

-1.1 

Grade 11 

Participants 

1,196 

77.1 

1,637 

80.6 

3.5 


Non- Participants 

4,039 

85.5 

3,784 

86.3 

0.4 

All Grades 

Participants 

4,553 

65.3 

5,641 

65.9 

0.6 


Non- Participants 

11,576 

76.2 

11,661 

76.8 

0.6 

Commended 

1 Performance 

Grade 8 

Participants 

1,717 

13.3 

1,923 

14.5 

1.2 


Non- Participants 

3,427 

22.1 

3,617 

25.2 

3.1 

Grade 10 

Participants 

1,640 

12.0 

2,081 

7.7 

-4.3 


Non- Participants 

4,110 

11.7 

4,260 

11.5 

-0.2 

Grade 11 

Participants 

1,196 

9.3 

1,638 

15.2 

5.9 


Non- Participants 

4,049 

11.8 

3,784 

16.2 

4.4 

All Grades 

Participants 

4,553 

11.8 

5,642 

12.2 

0.4 


Non- Participants 

11,586 

14.8 

11,661 

17.3 

2.5 


Sources: Texas Education Agency (TEA) R-Tech Student Upload data: fall 2008 and spring 2009. Public Education 
Information Management System (PEIMS) student demographic data from fall 2008. Texas Assessment of 
Knowledge and Skills 2008 and 2009 individual student data from TEA. 

Note. Students attended a campus participating in the R-Tech program in 2008-09. Participants took part in R-Tech 
instructional activities at one of the participating campuses. Non-participants attended an R-Tech participating 
campus but did not take part in R-Tech instructional activities. 
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TAKS Social Studies. Results for TAKS social studies in Table 6.4 show that, once again, participants 
had lower passing rates and commended performance rates than non-participants. The 2008 to 2009 
change in passing rates and commended performance rates were more positive for non-participants 
groups. The passing rate change was -1.2 percentage points for participant groups and -0.3 percentage 
points for non-participant groups. Likewise, the commended performance rate change was 3.9 percentage 
points for participant groups and 8.2 percentage points for non-participant groups. 


Table 6.4 

Cross-Sectional Longitudinal TAKS Passing and Commended Performance Rates for Social 
Studies: R-Tech Participants and Non-Participants 


TAKS Test Group 

2008 

N Percent 

2009 

N Percent 

2008 to 2009 
Difference 

Met Standard 

Grade 8 Participants 

1,710 

87.7 

1,758 

84.2 

-3.5 

Non-Participants 

3,411 

93.1 

3,448 

92.6 

-0.5 

Grade 10 Participants 

1,641 

88.2 

1,972 

87.5 

-0.7 

Non-Participants 

4,108 

90.9 

4,033 

91.0 

0.1 

Grade 11 Participants 

1,200 

96.3 

1,661 

95.8 

-0.5 

Non-Participants 

4,073 

97.0 

3,798 

96.9 

-0.1 

All Grades Participants 

4,551 

90.2 

5,391 

89.0 

-1.2 

Non-Participants 

11,592 

93.7 

11,279 

93.4 

-0.3 

Commended Performance 

Grade 8 Participants 

1,710 

26.5 

1,758 

26.5 

0.0 

Non-Participants 

3,411 

34.6 

3,448 

40.5 

5.9 

Grade 10 Participants 

1,641 

26.4 

1,972 

29.3 

2.9 

Non-Participants 

4,108 

30.1 

4,033 

38.2 

8.1 

Grade 11 Participants 

1,201 

29.7 

1,662 

38.5 

8.8 

Non-Participants 

4,074 

32.3 

3,798 

42.7 

10.4 

All Grades Participants 

4,552 

27.3 

5,392 

31.2 

3.9 

Non-Participants 

11,593 

32.2 

11,279 

40.4 

8.2 

Sources: Texas Education Agency (TEA) R-Tech Student Upload data: fall 2008 and spring 2009. 

Public 


Education Information Management System (PEIMS) student demographic data from fall 2008. Texas Assessment 
of Knowledge and Skills 2008 and 2009 individual student data from TEA. 

Note. Students attended a campus participating in the R-Tech program in 2008-09. Participants took part in R-Tech 
instructional activities at one of the participating campuses. Non-participants attended an R-Tech participating 
campus but did not take part in R-Tech instructional activities. 
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TAKS Writing. Table 6.5 shows that R-Tech participant groups had lower TAKS writing passing rates 
and commended performance rates than non-participant groups for each testing period. Changes between 
2008 and 2009 were more positive for the participating student groups. Their passing rate change was 1.6 
percentage points, while their commended performance rate change was 2.4 percentage points. 
Corresponding changes for non-participating groups were -1.4 percentage points and -0.7 percentage 
points, respectively. 


Table 6.5 

Cross-Sectional Longitudinal TAKS Passing and Commended Performance Rates for Writing: 
R-Tech Participants and Non- Participants 




2008 


2009 

2008 to 2009 

TAKS Test Group 

N 

Percent 

N 

Percent 

Difference 

Met Standard 

Grade 7 Participants 

1,567 

88.7 

1,950 

90.3 

1.6 

Non-Participants 

3,474 

94.4 

3,211 

93.0 

-1.4 

Commended Performance 

Grade 7 Participants 

1,567 

23.6 

1,950 

26.0 

2.4 

Non-Participants 

3,474 

36.9 

3,211 

36.2 

-0.7 


Sources: Texas Education Agency (TEA) R-Tech Student Upload data: fall 2008 and spring 2009. Public 
Education Information Management System (PEIMS) student demographic data from fall 2008. Texas Assessment 
of Knowledge and Skills 2008 and 2009 individual student data from TEA. 

Note. Students attended a campus participating in the R-Tech program in 2008-09. Participants took part in R-Tech 
instructional activities at one of the participating campuses. Non-participants attended an R-Tech participating 
campus but did not take part in R-Tech instructional activities. 

Summary of TAKS Achievement 

These data show that R-Tech participants had mostly lower TAKS passing rates and commended 
performance rates than non-participants. Passing and commended performance rate changes showed 
mixed results. In some cases, participants had larger gains (cohort longitudinal TAKS reading and 
mathematics passing rates). In other cases, non-participants had smaller losses (cohort longitudinal TAKS 
reading and mathematics commended performance rates). However, these changes arc difficult to 
interpret because participants and non-participants differed on initial levels of achievement (and may also 
have differed on socioeconomic background variables related to achievement). Because participants 
scored lower than non-participants, one would expect participants to have larger gains just because they 
had lower baseline scores. 2 Yet, participants did not consistently have larger gains than non-participants. 


“Because of imperfect (less than 1.0) TAKS score correlations across years, regression to the mean occurs. Thus, 
relatively low baseline scores tend to be associated with relatively high gains, irrespective of any improvement in 
student learning. Simply put, one would expect participants to have larger gains than non-participants just because 
they had lower baseline scores. 
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THE EFFECT OF INSTRUCTIONAL TIME, PROGRAM TYPE, AND SUPPLEMENTAL 
STATUS ON STUDENTS’ ACADEMIC ACHIEVEMENT 


The analysis of R-Tech’s effects on student achievement examines three aspects of the R-Tech program: 
(1) the effect of hours spent receiving R-Tech services on students’ TAKS scores, (2) whether R-Tech’s 
effects on students’ TAKS scores differed for supplemental programs and programs implemented as part 
of regular instruction, and (3) the effect of program type on students’ TAKS scores. The two major types 
of R-Tech programs implemented during the 2008-09 school year were self-paced computer software and 
dual credit or distance learning. 3 ' 4 The remaining three program types 5 were implemented in only two 
districts each. These programs are not included in analyses because of methodological limitations that 
arise when programs arc implemented in so few districts 

Methodology 

While TAKS measures mathematics, reading/ ELA, writing, science, and social studies, only mathematics 
and reading/ELA are tested at each grade level for Grades 6 through 1 1. Writing is only tested at Grade 7 
and science and social studies at Grades 8, 10, and 1 1. In order to incorporate a prior year TAKS test in 
the analyses, analyses were limited to Grades 6 through 1 1 in mathematics and reading/ELA and to Grade 
1 1 in both science and social studies. 

TAKS Scale Scores 

To measure students’ yearly progress, measurement specialists develop scales that may be used from one 
year to the next. These scales may be used to compare students’ performance across grade levels. Tests 
having these scales are called vertically equated tests. 6 The TAKS, however, is not a vertically equated 
test. 7 Both scale scores and percentages of items answered correctly vary across grades, and cannot be 
used to compare the performance of students at different grade levels. Because of this, researchers 
generated relative standard scores that were used to compare student progress on TAKS across grade 
levels. A standardized score — or z score — was calculated for each student and for every testing occasion 
and subject. The z score is calculated by subtracting the statewide mean grade-level scale score from each 
student’s scale score and dividing by the statewide scale score standard deviation. The z score, which has 
a mean of zero and a standard deviation of 1.0, indicates how many standard deviations from the mean a 
score lies. One characteristic of z scores is that about half of the scores are negative, and negative scores 
may be difficult to fully understand. To overcome this limitation, we have transformed students’ z scores 
into normalized scores, or T scores. T scores arc scores with a mean of 50 and a standard deviation of 10. 
Thus, a student who scores at the state average will have a TAKS T score of 50. A student who has a 
score of 60 will be one standard deviation above the state average, and a student who has a score of 40 
will be one standard deviation below the state average. 


initial categories of implementation (research-based instructional support, academic tutoring or counseling, distance 
learning aligned with the TEKS, distance learning to earn college credit, and web-based program) did not 
sufficiently describe districts' programs or distinguish between programs. See the chapter 3 for a discussion of the 
shortcomings of these categories. 

4 Program types were identified from an analysis of student upload information, district grant applications, and 
progress reports. Chapter 3 provides a detailed discussion of each program type. 

5 These program types were (1) live, person-to-person tutoring and online instructional support, (2) school-wide 
technology immersion, and (3) use of iPods to deliver content. 

6 When a test is vertically equated, a given score at one grade level of the test represents the same level of 
performance as that score on another grade level of the test. For example, with a vertically equated test, if the scale 
score is the same for a student in Grade 7 as it is for the student in Grade 8, one could conclude that the student did 
not make any progress in Grade 8. 

7 TEA is currently working on vertically equating TAKS. 
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Results 


Researchers formulated two sets of models. One set of models examined whether offering R-Tech 
services during regular school hours or outside of regular school hours (supplemental status) had an effect 
on students’ TAKS scores. The second set of models examined whether program type had an effect on 
students’ TAKS scores. The details of these analyses are reported in Appendix F, Tables F. 1 through 
F.12. In the supplemental status models, supplemental status was a significant negative predictor of social 
studies TAKS T scores. Supplemental status was also a non-significant negative predictor of students’ 
reading/ELA, mathematics, and science TAKS T scores. Thus, there is some evidence that offering 
R-Tech services during regular instructional hours or during elective periods may be more beneficial than 
offering services outside regular instructional hours. These supplemental status models also showed that 
the number of hours spent in R-Tech activities was not significantly related students’ TAKS scores. This 
means that time spent each week participating in R-Tech activities neither increased nor decreased TAKS 
T scores. 

In the program type models, R-Tech programs that utilized self-paced computer software were not 
significantly related to students’ TAKS mathematics, science, and social studies T scores. However, there 
was a significant negative relationship between self-paced computer software programs and students’ 
TAKS reading/ELA T scores. For example, the model predicted that students who used self-paced 
computer software programs scored 1.4 TAKS T score points lower in reading/ELA than students in other 
types of R-Tech programs. There was no significant relationship between dual credit or distance learning 
programs and students’ TAKS reading/ELA or mathematics T scores. As in the supplemental status 
models, the number of hours per week spent in R-Tech activities was not related to students’ TAKS 
scores. 

Several other interesting results were found across both sets of models. For example, school achievement 8 
was a significant positive predictor of TAKS mathematics T scores. Other factors being equal, R-Tech 
students performed better in mathematics when they were in schools with higher levels of achievement. 
These students benefitted from higher achieving peers. Other researchers have also shown that a schools’ 
achievement environment affects students’ academic achievement (Betts, Zau, & Rice, 2003; Chang, Wu, 
& Chen, 2007; Marks, McMillan, & Hillman, 2001). Results also indicate that female R-Tech students 
had higher spring 2009 TAKS reading/ELA T scores than males, after controlling for other predictors. 
However, males had higher spring 2009 TAKS social studies TAKS T scores than females, again, after 
controlling for other predictors. Net of the other predictor variables, African American students had lower 
spring 2009 TAKS reading/ELA T scores than non-minority students, economically disadvantaged 
students had lower spring 2009 TAKS reading/ELA and mathematics T scores than economically 
advantaged students, and LEP students had lower spring 2009 TAKS reading/ELA and mathematics T 
scores than non-LEP students. 

Limitations 

The goal of these analyses was not to generalize the results to all students who participated in R-Tech 
activities in all school districts. Rather, the puipose was to generalize to the target population of students 
who took part in R-Tech activities in the 115 participating campuses from the 62 participating school 
districts. Despite this rather modest aim, there were several potential limitations to the generalizability of 
findings. These limitations arc discussed in the sections that follow. 


8 School achievement was the percentage of students passing all 2009 TAKS tests at a campus. The percentages 
ranged 0.0% to 89.0%, with a grand mean of 64.8%. The 2009 school achievement data were preliminary results 
provided by TEA. 
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Missing data. Students were omitted from the analyses because of missing data. For example, varying 
numbers of students were missing demographic information (e.g., gender, ethnicity, economic status, and 
LEP status), reported R-Tech instructional hours in fall 2008 or spring 2009, TAKS scores in spring 2008 
or spring 2009, and campus supplemental status and program type data. (See Appendix F for a detailed 
description of the missing data issue.) As a result of missing data, original samples were reduced by 
42.5% for reading/ELA, 41.8% for mathematics, 32.3% for science, and by 30.9% for social studies. 

The characteristics of students having complete data (restricted samples) were compared to the 
characteristics of the original samples. Gender differences between the two samples were slight as were 
ethnic differences for the reading/ELA and mathematics samples. The restricted science and social studies 
samples had lower percentages of minority students. The restricted reading/ELA, mathematics, and 
science samples had lower percentages of LEP students. All of the restricted samples had lower 
percentages of economically disadvantaged students and especially special education students. Thus, the 
restricted samples were different than the original samples. They were less likely to be economically 
disadvantaged, LEP, and special education students. 

TAKS as the measure of student achievement. Because the TAKS is not a vertically equated test (i.e., 
the skills measured and the scoring from one grade to the next is along a continuum), results arc not 
comparable from grade to grade and from year to year. Thus, researchers used standard scores (T scores) 
to compare students from one year to the next. These scores allow for normative comparisons (where 
students fall in the distribution of test scores from one year to the next), but not for criterion-referenced 
comparisons (where students fall on a proficiency scale from one year to the next). In addition, TAKS 
does not measure science and social studies at each of the tested grade levels. Thus, to include a prior year 
test, science and social studies samples included only Grade 1 1 students. This resulted in relatively small 
numbers of students in the science and social studies analyses. 

SUMMARY 

This chapter compared the scores of cohorts of participating and non-participating students who took the 
TAKS test in both spring 2008 and spring 2009. For all tested areas, R-Tech participants had lower TAKS 
passing rates and commended performance rates than non-participants. Because participants scored lower 
than non-participants, one would expect participants to have larger gains because of lower baseline 
scores. Flowever, participants in R-Tech did not consistently have larger gains than non-participants. 

This chapter also investigated the effects of three aspects of R-Tech program implementation on students’ 
TAKS scores. These aspects were student access time, supplemental status, and program type. 

Researchers found that there was not a significant relationship between student access time or reported 
R-Tech instructional hours per week and students’ TAKS scores. Supplemental status, or offering 
programs outside regular school hours, had a significant negative relationship to students TAKS social 
studies scores, and a non-significant negative relationship to students’ TAKS reading/ELA, mathematics, 
and science T scores. Self-paced computer software programs had a significant negative relationship to 
students’ TAKS reading/ELA scores, but no significant relationship to students’ TAKS mathematics, 
science, and social studies T scores. There was no significant relationship between dual credit or distance 
learning programs and students’ TAKS reading/ELA and mathematics scores. 

These results may not generalize to R-Tech students who did not maintain continuous enrollment or to 
students for whom the TAKS test is not an appropriate measure of performance (i.e., special education 
students). Construction of data files used in the achievement analyses resulted in reduced numbers of 
students. These restricted samples differed from the overall population of R-Tech students, as they were 
less likely to be economically disadvantaged, LEP, and special education students. 
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Chapter 7 

The Cost Effectiveness of R-Tech 


Given the need to use educational resources efficiently, policymakers are increasingly requiring cost- 
effectiveness analyses of educational interventions to inform their decision making and to ensure that 
greatest “bang for the buck” in terms of spending on public education (Levin & McEwan, 2001). As a 
means to ensure Texas’ educational resources are spent in an efficient manner, state legislators required 
that R-Tech be evaluated in terms of its cost effectiveness, as well as its effect on academic outcomes. 

Generally speaking, cost-effectiveness analysis seeks to evaluate policy alternatives relative to their costs 
in producing a given outcome (Levin & McEwan, 2001). However, in the case of this evaluation, the only 
program under consideration is R-Tech. Understanding that the measurement of R-Tech’ s cost 
effectiveness would not include comparable programs, the evaluation frames its analysis in terms of how 
different approaches to implementing R-Tech affect the program’s cost effectiveness. In assessing 
R-Tech’s cost effectiveness the evaluation seeks to answer the following questions: 

• How are R-Tech funds allocated in districts? 

• Which R-Tech program configurations make the most effective use of funding? 

• Will the R-Tech program be sustained after grant funds expire? 

To answer these questions, this chapter draws on data collected through TEA’s ER system, which 
includes information about how districts spend state -provided R-Tech funds, project budgets included in 
grant applications, and TEA data documenting individual district grant awards. The chapter also uses 
information about the number of students served by R-Tech reported in district upload data provided to 
TEA for summer 2008, fall 2008, and spring 2009. In addition, the chapter incorporates information 
drawn from document analysis of R-Tech grant applications and progress reports, interviews with 
principals in eight R-Tech districts, as well as the spring 2009 survey of principals in all grantee districts. 1 

The findings presented in this chapter provide preliminary information about R-Tech’s cost effectiveness 
and sustainability. The chapter addresses how R-Tech districts have used program funding through spring 
2009, provides preliminary estimates of the per-student cost of R-Tech services for all R-Tech districts 
and for districts implementing particular types of programs, and considers principals’ views of program 
sustainability. The chapter does not examine R-Tech’s cost effectiveness in relation to student 
achievement outcomes. 2 

OVERVIEW OF R-TECH FUNDING 

The Texas Legislature provided for $8 million in funding for R-Tech. TEA awarded about $6.3 million in 
Cycle 1 grants, and an additional $1.5 million in Cycle 2 grants. Grantee districts receive $200 per school 
year in state grant funding for each student receiving R-Tech services and are required to provide at least 
$100 per participating student per school year in matching funds. Matching funds may consist of local or 
private funding or from state funds other than those provided by the R-Tech grant, and districts may use 
HSA monies to provide matching funds at the high school level. The Cycle 1 grant period runs from May 
1, 2008, to May 31, 2010, and the Cycle 2 grant period runs from January 1, 2009, to May 31, 2010. 


'information about the spring 2009 principal survey, including administration processes, response rates, and a copy 
of the survey, may be found in Appendix A. Information about site visits may be found in Appendix D. 

2 The final evaluation report (fall 2010) will analyze program costs in relation to student achievement outcomes, such 
as TAKS scores and graduation rates. 
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Grantee districts must budget their use of R-Tech funds within these dates. Permissible grant expenditures 
include costs incurred to provide: 

• Research-based instructional support, 

• Teacher training, 

• Academic tutoring or counseling, 

• Distance learning opportunities in the core content areas or in foreign languages, and 

• Dual credit coursework in the core content areas or in foreign languages. 

Because the evaluation focuses on districts that received Cycle 1 R-Tech grants, this chapter only 
addresses districts’ use of Cycle 1 funding. 

TEA initially awarded $6,384,743 in Cycle 1 R-Tech funding to 64 rural Texas districts. One district with 
an award of $60,000 withdrew from the program without accessing funding, which reduced the total grant 
award to $6,324,743. R-Tech Cycle 1 funding is awarded across two fiscal years (FY): FY 2008 
(September 1, 2007-August 31, 2008) and FY 2009 (September 1, 2008-August 31, 2009). Sixty-two 
percent of grant funding was allocated to FY 2008 ($3,912,293), and the remainder allocated to FY 2009 
($2,412,450). Table 7.1 presents the minimum, maximum, total, and average grant awards across fiscal 
years and for the total Cycle 1 grant award. In order to ensure that a wide range of districts would have 
access to R-Tech funding, TEA established a maximum grant award of $200,000. The Agency did not 
specify a minimum award amount. As indicated in the table, R-Tech grant awards ranged from $16,000 to 
$200,000, and the average grant award was about $100,393. 


Table 7.1 

The Structure of Cycle 1 State-Level R-Tech Grant Funding 


Award 

FY 2008 

FY 2009 

Cycle 1 Total 

Minimum award 

$10,055.00 

$5,945.00 

$16,000.00 

Maximum award 

$120,886.00 

$79,114.00 

$200,000.00 

Total awards 

$3,912,293.00 

$2,412,450.00 

$6,324,743.00 

Average award 

$62,099.89 

$38,292.86 

$100,392.75 


Source: Texas Education Agency (TEA) grant award documents. 

Note. TEA established a maximum grant award of $200,000; it did not set a minimum award amount. 


State grant funds are available to R-Tech districts through TEA’s ER system, through which districts 
report grant expenditures and request reimbursements. Grant requirements establish that districts request 
funding “as close as possible to the time of making disbursements,” and ask that grantees only request 
funding for expenditures that will be made within 3 days of a payment (TEA, 2008b). When requesting 
R-Tech payments, districts report expenditures in terms of budgeting categories identified in their grant 
applications (e.g., payroll, supplies and materials). TEA denies payment for funds requested for categories 
not included in a district’s grant application, when the requested funding exceeds the allowable budget 
variation for a category, and when the total amount requested by a district exceeds its grant. Districts may 
only enter up to 90% of their cumulative expenditures in ER. When a district has met its final reporting 
requirements, 3 TEA releases the remaining 10% of the grant award and requests that the district submit a 
revised final expenditure report. 


districts' final expenditure reports must be filed within 30 days of the grant’s ending date. 


94 





LIMITATIONS OF THE ANALYSIS 


The cost-effectiveness analysis presented in this chapter encounters some limitations that arise from the 
available data sources, from valuations in districts’ implementation plans and access to state grant funding 
across the program’s first year, and from differences in districts’ budgeting practices. The sections that 
follow discuss data sources and the limitations of analyses. 

Data Sources 

Perhaps the greatest limitation to the evaluation’s analysis of R-Tech’s cost effectiveness is its reliance on 
financial data provided through district expenditure reports and project budgets reported in district grant 
applications. In Cost-Effectiveness Analysis (2001), authors Levin and Me Ewan argue that analyses of 
project budgets and expenditure documents are “inadequate” to assess the “true” costs of an educational 
intervention such as R-Tech. The authors note that budgets do not capture the costs of “unpaid” resources 
such as volunteers and donated equipment, and that budgets do not include the costs of resources that 
were purchased prior to the implementation of an intervention. In the case of R-Tech, such resources may 
include computer equipment, lab space, printers, technology infrastructure, and teacher training purchased 
prior to and irrespective of R-Tech. Levin and McEwan further argue that budgets and expenditure reports 
“distort” the true costs of resources because costs are recorded for a particular year, but some resources 
are used well beyond the year in which costs are recorded. For example, computers and software 
purchased with R-Tech funding may be recorded in a district’s budget for FY 2008, but the equipment 
also will be used in years to come, which overstates their costs in FY 2008 and understates their costs in 
subsequent years (pp. 45-46). 4 

A further limitation arises because the expenditure reports districts provide to TEA only document how 
districts use state grant funding and do not include information on how district matching funds are spent. 
Therefore, the chapter’ s analyses arc limited to the use of state funding, and do not include information 
about district-level funding for R-Tech. While these limitations prevent researchers from assessing the 
“true” costs of R-Tech in terms of total project expenditures, the costs of pre-existing project resources, 
and the discounting of equipment that schools will use well beyond the R-Tech grant period, the available 
data sources do allow researchers to assess the state’s costs in implementing R-Tech, which is of concern 
to state-level policymakers interested in how educational resources are used. 

Variations in Districts Approach to Funding and Implementing R-Tech 

Variations in districts’ implementation plans and approaches to accessing R-Tech funds across the 
program’s first year limit the second interim report’s ability to fully answer the evaluation’s questions 
about R-Tech’s cost effectiveness during the program’s first year. These valuations underscore the 
preliminary nature of this chapter’s analyses. The final evaluation report (fall 2010) will document 
districts’ use of total grant funding when program implementation is largely complete and will provide 
more conclusive evidence of R-Tech’s cost effectiveness. 

Variations in access to state grant funds. Although R-Tech funding was allocated by fiscal year, 
grantee districts were not required to use funding within specified fiscal years. In May 2009, the cutoff 
point for expenditures analyzed in this report. Cycle 1 districts had used about 50% of total grant funding. 


4 To overcome the limitations of budget and expenditure documents. Levin and McEwan (2001) recommend an 
approach to cost-effectiveness analysis that requires the identification of the specific “ingredients” used to 
implement an intervention, including personnel, facilities, equipment and supplies, and so on. They outline a process 
for establishing the “value” of individual ingredients through the analysis of market prices, subsidies, depreciation 
costs, discount rates, and adjustments for inflation. Such an analysis is beyond the scope of this evaluation. 
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accessing 70% of FY 2008 funds and 19% of FY 2009 funds. 5 Five Cycle 1 districts had not accessed any 
state grant funding in May 2009. Across the 58 districts that had accessed FY 2008 funds, the percentage 
of funds used ranged from 3% to 100%. For districts that had accessed FY 2009 funding, the amount of 
FY 2009 funds used ranged from 6% to 90%. In terms of total grant funding, the percentage of funds used 
by the 58 districts accessing state funding ranged from 2% to 96% in May 2009. 6 

Variations in implementation levels. As indicated in the previous section, five districts had not accessed 
any of their state grant awards in spring 2009, although four of these districts had students who received 
R-Tech services during the program’s first year. In contrast, five districts had not served any students 
across the summer 2008, fall 2008, or spring 2009 reporting periods, but four of these districts had spent 
grant funding, accessing from $33,078 to $59,326 of state funding. Analyses presented in previous report 
chapters included 62 R-Tech districts, noting that one district had been omitted because it deferred 
implementation of its program to the 2009-10 school year. This district is included in discussions of cost 
effectiveness, however, because it accessed funding during R-Tech’ s first year. Therefore, the number of 
districts included in the chapter’s analyses is 63. 

Inconsistency Across Grant Application Budget Categories 

Another limitation arises because districts were not consistent in how they classified planned expenditures 
in their grant applications. Grant applications enable districts to plan expenditures in terms of (1) payroll 
costs, (2) professional and contracted services, (3) supplies and materials, (4) other operating costs, and 
(5) capital outlay. Flowever, districts classified expenditures differently across categories. For example, 
some districts included software purchases in “professional and contracted services,” while others 
categorized software purchases as “supplies and materials,” and some reported software purchases as 
“capital outlay.” Similarly, some districts budgeted computer hardware in the “supplies and materials” 
category, while others considered hardware to be “capital outlay.” Districts also use these budget 
categories to report expenditures in TEA’s ER system, and the lack of consistency in district budgets 
limits the comparisons that may be made across the expenditure categories discussed in the chapter’s next 
section. 

DISTRICT ALLOCATION OF R-TECH FUNDS 

The cost-effectiveness analysis seeks to understand how districts allocate their funds in implementing 
R-Tech. To address this question, researchers examined districts’ expenditure patterns across the budget 
categories identified by TEA’s ER system. Similar to grant application budget categories, ER classifies 
expenditures as (1) payroll costs, (2) professional and contracted services, (3) supplies and materials, (4) 
other operating costs, and (5) capital outlay. Each budget category is further divided into expenditures for 
program costs and those for administrative costs. Program costs include those expenditures directly 
related to implementing R-Tech, while administrative costs support functions related to project 
management and administration. Grant requirements limit the amount of R-Tech funding that may be 
spent on administrative costs to 5% of a district’s total grant award in any fiscal year. Administrative 
funds must be used for administrative costs directly related to R-Tech, including salaries for staff who 
supervise R-Tech and costs associated with project accounting. 


5 Sixteen districts had used all of their FY 2008 funding and had begun to access FY 2009 funds in May of 2009. 

6 Although grant requirements limit districts to 90% of cumulative state funding prior to the close of the grant, in the 
case of R-Tech, cumulative funding was understood as the FY 2009 grant award. That is, districts that accessed 96% 
of total funding had accessed only 90% of FY 2009 funding. 
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Total R-Tech Expenditures by Budget Category 

Table 7.2 presents the total ER-recorded expenditures for R-Tech districts as of May 2009. Results 
indicate that most R-Tech funds (about $1.8 million) were used to purchase supplies and materials for the 
grant. Smaller amounts of R-Tech funding were used for professional and contracted services (about 
$419,000), capital outlay (about $276,000), and payroll costs (about $227,000). The sections that follow 
provide information about the types of expenditures included in each cost category. 


Table 7.2 

Total District First Year Grant Expenditures, by R-Tech Funding Categories: May 2009 


Cost Category 

Program Costs 

Administrative 

Costs 

Total Costs 

Payroll costs 

$207,362.98 

$19,650.65 

$227,013.63 

Professional and contracted services 

$415,910.29 

$2,966.00 

$418,876.29 

Supplies and Materials 

$1,848,282.24 

— 

$1,848,282.24 

Other operating costs 

— 

— 

— 

Capital outlay 

$275,784.52 

— 

$275,784.52 

Total Costs 

$2,747,340.03 

$22,616.65 

$2,769,956.68 


Source: Texas Education Agency Expenditure Reporting system data. May 2009. 
Note. N=63. 


Payroll costs. Payroll costs include expenditures for school employees and non-employees (e.g., 
consultants) who spend all or some of their time working on R-Tech activities. In grant applications, 
R-Tech districts indicated that payroll costs would be spent for extra-duty pay for school employees who 
work additional hours (e.g., before or after school) to provide R-Tech services, salaries for R-Tech 
facilitators and computer lab staff, as well as the costs of substitutes to enable teachers to participate in 
training. In May 2009, 8% of state -provided R-Tech funding had been used to cover payroll costs related 
to the grant. Across districts, payroll expenditures ranged from $0 to $25,641.48. 

Professional and contracted services. The professional and contracted services category includes the 
costs of educational software, registration fees and tuition for online courses, professional development 
and training in the use of software, technical support services, and fees for services provided by ESCs in 
support of R-Tech. Fifteen percent of R-Tech expenditures in May 2009 had been used for professional 
and contracted services. Across districts, expenditures on professional and contracted services ranged 
from $0 to $69,249.79. 

Supplies and materials. About two thirds (67%) of R-Tech expenditures had been spent on supplies and 
materials in May 2009. This budget category was used to pay for laptop and desktop computers, printers, 
LCD projectors, scanners, computer lab furnishings (e.g., tables and chairs), dual display video 
equipment, textbooks for dual credit courses, and some districts budgeted computer software in this 
category. District expenditures on supplies and materials ranged from $0 to $120,866.00. 

Other operating costs. Other operating costs reported in district grant application budgets included travel 
expenses for staff and students participating in R-Tech and printing costs. R-Tech districts did not report 
any “Other Operating Costs” in May 2009. 

Capital outlay. Capital outlay funds may be used to purchase nonexpendable, tangible, personal property 
with a useful life of more than 1 year. In grant application budgets, most districts categorized purchases of 
R-Tech computer hardware and software in the “supplies and materials” funding category; however, some 
districts recorded these purchases as “capital outlay.” About 10% of districts expenditures were recorded 
as capital outlay in May 2009. District expenditures on capital outlay ranged from $0 to $85,457.52. 
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Average Grant Expenditures and the Allocation of R-Tech Funds by Program Type 

Table 7.3 presents average district expenditures across all districts. Noting the small proportion of funding 
allocated to administrative costs (0.8%), the comparisons of average expenditures by program type that 
follow focus on districts’ total costs for each expenditure category and incorporate information from the 
“Total Costs” column in Table 7.3 as a benchmark for understanding whether program expenditures arc 
above or below average relative to all R-Tech programs. 


Table 7.3 

Average District First Year Grant Expenditures, by R-Tech Funding Categories: May 2009 


Cost Category 

Program Costs 

Administrative 

Costs 

Total Costs 

Payroll costs 

$3,291.47 

$311.92 

$3,603.39 

Professional and contracted services 

$6,601.75 

$47.08 

$6,648.83 

Supplies and Materials 

$29,337.81 

— 

$29,337.81 

Other operating costs 

-- 

— 

— 

Capital outlay 

$4,377.53 

- 

$4,377.53 


Source: Texas Education Agency Expenditure Reporting system data. May 2009. 


In order to compare district’s use of R-Tech funds across the types of R-Tech programs the sections that 
follow examine district’s average expenditures across cost categories. 

Types of instructional programs. Researchers examined district grant applications and progress reports 
in order to categorize districts in terms of the types of instructional programs offered through R-Tech. 
This relies on the five general categories of R-Tech programs described in chapter 3: (1) self-paced 
programs focused on remediation, tutoring, or credit recovery; (2) dual credit and distance learning 
opportunities; (3) one-to-one support from tutors complemented by online instructional support; (4) 
school-wide technology immersion projects; and (5) the provision of iPods that enable students to access 
targeted instructional support (e.g., ELL programs). The categories are not discrete — districts may be 
included in more than one category. This occurs because R-Tech permits districts to implement different 
types of programs at the middle and high school levels. Lor example, a district may choose to offer dual 
credit coursework at its high school, but offer a self-paced program at its middle school. While middle 
schools and high schools may implement different types of programs, expenditures are reported at the 
district level, which prevents the identification of funds spent for middle school and high school 
programs. 

Table 7.4 presents districts’ average costs (administrative and program costs) by each of the five R-Tech 
program categories discussed in the previous paragraph. Results indicate that the ways in which districts 
choose to implement R-Tech affects how grant funds are spent. Perhaps most notable is the large amount 
spent on professional and contracted services for districts implementing dual credit/distance learning 
programs and districts implementing one-to-one tutoring and support, as well as the amount spent on 
supplies and materials by districts implementing technology immersion programs. 
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Table 7.4 

R-Tech Average District First Year Grant Expenditures, by Program Type and Funding 
Categories: May 2009 


Program Type 

Number of 
Districts in 
Category 

Payroll 

Costs 

Professional 

and 

Contracted 

Services 

Supplies and 
Materials 

Capital 

Outlay 

Self-paced software 

54 

$3,606.96 

$6,651.69 

$29,829.51 

$4,442.66 

Dual credit/distance learning 

18 

$4,200.75 

$12,095.28 

$19,626.84 

$1,482.29 

One-to-one tutoring and 
online instructional support 

2 

$955.00 

$52,715.00 

$0.00 

$0.00 

Technology immersion 

2 

$2,583.50 

$3,150.00 

$67,278.50 

$0.00 

iPods 

2 

$425.00 

$2,055.00 

$7,099.50 

$0.00 

All district programs 

63 

$3,603.39 

$6,648.83 

$29,337.81 

$4,377.53 


Sources: Texas Education Agency Expenditure Reporting system data. May 2009; district grant applications and 
progress reports. 


Some districts implementing dual credit and distance learning programs contracted with external vendors 
to provide technology-based instructional programs, which explains their higher than average 
expenditures on professional and contracted services. Districts implementing one-to-one tutoring 
programs with online instructional support spent nearly 8 times as much on contracted services relative to 
the average for all R-Tech programs ($51,715 vs. $6,648). Both districts implementing this type of 
program contracted with TxRED to provide R-Tech services. 7 TxRED offers online dual credit courses, 
distance learning opportunities, online tutoring and credit recovery, and a range of technology-based 
professional development programs in conjunction with the University of Texas at Austin’s K-16 
program. Both districts contracting with TxRED offered dual credit programs at the high school level and 
one-to-one tutoring and online support for middle school students. In terms of their total budgets, the 
districts had allocated an average of $81,130 in state funding for TxRED services and had accessed an 
average of $52,715 to pay for services through May 2009, although neither district reported that students 
received R-Tech services during the program’s first year. 

Districts that implemented R-Tech as a school-wide technology immersion program spent more than 
twice the program average on supplies and materials ($67,278 vs. $29,337). This result is due to one 
district that budgeted its full grant award ($200,000) to purchase laptops for all teachers and students in 
Grades 6 through 12. The district had accessed 60% of its grant award ($120,886) in May 2009, which 
inflated the average expenditures for supplies and materials across the two districts. 

Supplemental vs. non-supplemental programs. Although R-Tech funding was intended to support 
supplemental programs offered outside of the regular school day (e.g., before or after school), many 
grantee districts (40%) implemented R-Tech as part of the regular instructional day. For example, some 
districts used R-Tech funding to purchase laptop computers and software that teachers incorporated as 
part of daily instruction. In other districts, R-Tech funding supported improvements to computer labs and 
provided educational software that students accessed when teachers scheduled class time in the lab. Using 
descriptions of R-Tech programs provided in grant applications and progress reports, researchers 
categorized programs as supplemental or non-supplemental, depending on when students accessed 
R-Tech services (i.e., at a time outside of regularly scheduled classes or during regular class time). 


7 For more information about TxRED, visit the Consortium’s web site at http://txred.org. 
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Table 7.5 presents average expenditures for districts that implemented R-Tech as a supplemental program 
and for those that incorporated R-Tech into the regular school day. Results indicate that supplemental and 
non- supplemental programs spent similar amounts, on average, for payroll costs. On average, 
supplemental programs spent about $880 more on professional and contracted services, but non- 
supplemental programs spent about $1 1,000 more on supplies and materials and about $5,000 more on 
capital outlay. Non-supplemental programs’ expenditures on supplies and materials and capital outlay 
also exceeded the averages for all district programs by about $7,500 and $2,200, respectively. 


Table 7.5 

R-Tech Average District First Year Grant Expenditures, by Supplemental and Non-Supplemental 
Implementation and Funding Categories: May 2009 


Program Type 

Number of 
Districts in 
Category 1 * 

Payroll Costs 

Professional 

and 

Contracted 

Services 

Supplies and 
Materials 

Capital 

Outlay 

Supplemental programs 

34 

$3,310.40 

$7,418.56 

$25,687.10 

$1,576.94 

Non-supplemental programs 

25 

$3,330.89 

$6,538.96 

$36,890.93 

$6,625.26 

All district programs 

63 

$3,603.39 

$6,648.83 

$29,337.81 

$4,377.53 


Sources: Expenditure Reporting system data. May 2009; district grant applications and progress reports. 

‘Tour districts are not included in either category because it was not possible to determine from grant applications 
and progress reports whether their programs were supplemental or non-supplemental. 


The findings presented in Table 7.5 suggest that non-supplemental programs used substantially more state 
funding to purchase supplies and equipment for their R-Tech programs. It is likely that districts that 
implemented R-Tech as paid of the regular school day did so for larger numbers of students, and 
therefore, needed to purchase considerably more resources than districts that implemented supplemental 
programs. This finding is explored in more detail in the next section. 

THE COST EFFECTIVENESS OF R-TECH FUNDING 

The cost-effectiveness findings presented in the following sections are preliminary and are not linked to 
student outcomes, such as TAKS scores and attendance rates. As discussed earlier in this chapter, districts 
varied in their approaches to implementing R-Tech across the program’s first year. Some districts used 
their first year to plan R-Tech services, improve infrastructure, and to purchase equipment, while others 
fully implemented the program for students. These variations combined with heavy start-up investments 
in technology resources may distort per-student calculations of districts' use of state funding during 
R-Tech’ s first year. As explained earlier in this chapter, investments in equipment, such as computer 
hardware and software, will benefit districts for years to come, and analyses of district expenditures in 
terms of students served in R-Tech’ s first year will overstate R-Tech’ s per-student costs. These 
limitations will be offset to some extent in the evaluation’s final report (fall 2010), which will examine 
costs across both years of R-Tech implementation, including all students served with grant funds. 
However, it is likely that equipment purchased with grant dollars will benefit students who do not 
participate in R-Tech as well as students enrolled in districts after grant funds expire, which limits the 
evaluation’s ability to estimate the “true” per-student costs of R-Tech implementation. 

Acknowledging the uneven implementation across the program’s first year, the overstatement of first-year 
costs for technology resources, and researchers’ inability to assess the true per-student costs of R-Tech, 
the cost-effectiveness findings presented in the sections that follow present preliminary analyses of the 
per-student costs of implementing R-Tech services by the size of R-Tech program, the type of program 
implemented, and whether districts implemented supplementary or non-supplementary programs. The 
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student counts used in analyses arc the number of unique students served by R-Tech across the summer 
2008, fall 2008, and spring 2009 reporting periods. Students are counted as receiving R-Tech services if 
they participated in R-Tech during any of the three periods, and students who received services across 
multiple periods are included only once in counts. 

The Scale of R-Tech Programs 

Program size matters when estimating the per-student cost of educational interventions. Generally 
speaking, programs that are implemented more broadly and serve larger numbers of students experience 
economies of scale that enable them to enjoy lower per-student program costs. Levin (2002) explains: 

At lower enrollments the cost per student will be high because the fixed costs must be divided 
among a very small number. However, with larger enrollments the fixed costs do not rise 
commensurately so that average costs per student drops. Therefore, the comparison of costs must 
be sensitive to different levels of scale rather than relying on a single enrollment level to estimate 
costs (p. 26). 

Table 7.6 presents information about the average number of students served, average expenditures, and 
average state-funded expenditures per student across districts grouped by scale of implementation. The 
table presents findings broken out by the total number of students served by R-Tech programs, 
categorizing districts as serving 0 students, 1 to 49 students, 50 to 99 students, 100 to 249 students, and so 
on. 


Table 7.6 

Preliminary Per-Student State-Funded R-Tech Expenditure Calculations, by the Number of 
Students Served: May 2009 


Students Served: 
Summer 2008, Fall 
2008, Spring 2009“ 

Number of 
Districts in 
Category 

Average Number of 
Students Served 

Average 

Expenditures 

Average 
Expenditures 
per Student 

0 

5 

Undefined 

$35,390.32 

Undefined 

1-49 

9 

27.9 

$12,811.81 

$1,521.43 

50-99 

12 

75.6 

$19,349.62 

$271.09 

100-249 

13 

151.3 

$34,828.79 

$244.37 

250-500 

14 

331.6 

$72,747.62 

$221.65 

500 or more 

10 

708.0 

$77,426.24 

$111.31 

Total 

63 

235.7 

$43,967.57 

$419.64” 


Sources: Texas Education Agency (TEA) Expenditure Reporting system data. May 2009; TEA Student Upload data, 
summer 2008, fall 2008, spring 2009. 


Notes. An undefined ratio occurs when 0 is the denominator. 

“Students included in the analysis received R-Tech services in at least one of the following periods: summer 2008, 
fall 2008, and spring 2009. Students receiving services across multiple periods are counted only once in the analysis. 
b Total average expenditures per student were calculated using only those districts that served R-Tech students during 
the summer 2008, fall 2008, and/or spring 2009 (n=58). 

Given differences in how districts implemented R-Tech during the grant’s first year, it is not surprising 
that there are wide variations across districts in the numbers of students served, average expenditures, as 
well as in average expenditures per student. Beyond expected variations, the information provided in 
Table 7.6 provides insight into how districts’ implementation strategics may affect the overall cost 
effectiveness of R-Tech. The table demonstrates the economies of scale that exist for districts that 
implemented R-Tech broadly in the program’s first year — while average expenditures increased when 
more students participated in the program, districts experienced a corresponding decrease in average costs 
per student when more students accessed services. Districts serving 500 or more students experienced the 
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highest average costs ($77,426), but experienced per-student costs ($111) that were less than half that of 
most districts serving fewer students during R-Tech’s first year. 

Per-Student Costs by Type of Instructional Program 

The cost-effectiveness analysis also seeks to understand how different approaches to implementing 
R-Tech may affect program costs. As discussed earlier in this chapter, researchers identified five non- 
discrete categories of R-Tech program configurations and categorized programs as providing 
supplemental vs. non-supplemental instruction. 

Table 7.7 presents the average number of students served, average expenditures, and average expenditures 
per student for each R-Tech program configuration identified by this evaluation, as well as for all districts 
implementing R-Tech. Because neither of the districts offering one-to-one tutoring and online support 
served students during R-Tech’s first year, the average per-student cost for this program category is 
undefined. Results indicate that the per-student costs of implementing R-Tech programs varied widely 
across R-Tech’s first year. Variations reflect differences in the types of start-up costs districts experienced 
in implementing programs, as well as the number of students served. Districts offering self-paced 
remediation, tutoring, and credit recovery programs experienced the highest per-student costs ($429), 
which is likely the result of substantial start-up investments in technology resources. Table 7.4 in the 
previous section indicated that districts implementing self-paced programs spent about $34,000, on 
average, on supplies and equipment and capital outlay across their first year of implementation, and as 
noted earlier in this chapter, most districts included purchases of computer hardware and software in these 
expenditure categories. As districts serve more students with equipment purchased using grant funds, 
average expenditures per student are expected to drop. Similarly, districts implementing technology 
immersion programs experienced high start-up costs because of initial investments in technology 
resources. Technology immersion programs experienced the highest average costs of all program types 
($73,012), but per-student average expenditures were considerably lower than those of self-paced 
programs ($269 vs. $429) because technology immersion programs were implemented more broadly, 
serving more than 300 students, on average, in R-Tech’s first year. Again, as programs expand to serve 
more students over the second year of the grant, average expenditures per student are expected to drop 
substantially. 
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Table 7.7 

Preliminary Per-Student State-Funded R-Tech Expenditure Calculations, by Program Type: May 
2009 


Program Type 

Number of 
Districts in 
Category 

Average Number of 
Students Served 

Average 

Expenditures 

Average 
Expenditures 
per Student 

Self-paced software 

54 

256.0 

$44,530.82 

$428.57 

Dual credit/distance 
learning 

18 

176.8 

$37,405.15 

$197.98 

One-to-one tutoring and 
online instructional support 2 

2 

0.0 

$53,670.00 

Undefined 

Technology immersion 

2 

305.5 

$73,012.00 

$269.22 

iPods 

2 

55.0 

$9,579.50 

$358.32 

Total 

63 

235.7 

$43,967.57 

$419.64 b 


Sources: Texas Education Agency (TEA) Expenditure Reporting system data. May 2009; TEA Student Upload data, 
summer 2008, fall 2008, spring 2009; district grant applications and progress reports. 

Notes. Program type categories are not discrete and will not total to 63; districts may be included in more than one 
category. Averages are averages across districts’ total expenditures and per-student costs. An undefined ratio occurs 
when 0 is the denominator. 

“No district offering one-to-one tutoring and online instructional support served any R-Tech students in summer 
2008, fall 2008, or spring 2009. 

b Total average expenditures per student were calculated using only those districts that served R-Tech students during 
the summer 2008, fall 2008, and/or spring 2009 (n=58). 

Programs that required smaller investments in technology resources to get started experienced lower 
average costs during R-Tech’ s first year. While dual credit and distance learning programs spent less on 
average on supplies and materials, including technology resources, than other R-Tech programs, they 
tended to spend more on contracted services relative to other districts (see Table 7.4). This trend suggests 
that dual credit and distance learning programs may not experience a reduction in per-student costs across 
implementation years because, unlike purchases of technology equipment, resources spent on contracted 
services will not last beyond the contract period. 

Supplemental vs. Non-Supplemental Implementations Per-Student Program Costs 

As noted earlier in this chapter, about 40% of districts receiving R-Tech funding implemented the 
program as part of the regular school day rather than as a supplemental activity offered outside of 
students’ regularly scheduled classes. Table 7.8 presents information about the average number of 
students served, average expenditures, and average expenditures per-student for districts offering 
supplemental and non-supplemental instruction. Results for all districts are presented for putposes of 
comparison. 
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Table 7.8 

Preliminary Per-Student State-Funded R-Tech Expenditure Calculations, by Supplemental and 
Non-Supplemental Instruction: May 2009 


Program Type 

Number of 
Districts in 
Category 

Average Number of 
Students Served 

Average 

Expenditures 

Average 
Expenditures 
per Student 

Supplemental programs 

34 

171.6 

$37,992.99 

$612.30 

Non-supplemental programs 

25 

349.8 

$53,386.05 

$187.04 

Total 

63 

235.7 

$43,967.57 

$419.64“ 


Sources: Texas Education Agency (TEA) Expenditure Reporting system data. May 2009; TEA Student Upload data, 
summer 2008, fall 2008, spring 2009; district grant applications and progress reports. 

Notes. Four districts are not included in either category because it was not possible to determine from grant 
applications and progress reports whether their programs were supplemental or non-supplemental. 
a Total average expenditures per student were calculated using only those districts that served R-Tech students during 
the summer 2008, fall 2008, and/or spring 2009 (n=58). 

Findings presented in Table 7.8 indicate that districts that implemented R-Tech as part of the regular 
school day (i.e., non-supplemental programs) experienced substantially lower per-student costs relative to 
districts that implemented supplemental programs ($187 vs. $612), as well as to R-Tech districts overall 
($187 vs. $419). The reduced per-student costs for non-supplemental programs result from districts 
broader implementation of R-Tech. On average, districts that offered R-Tech services as part of the 
regular school day served 350 students during the first year of implementation. In contrast, districts 
offering supplemental programs served 172 students, on average. 

SUSTAINABILITY 

A central evaluation question is whether districts will continue to provide R-Tech services after grant 
funds expire in the spring 2010. The spring 2009 survey of R-Tech principals included questions about 
the barriers to sustaining the program and strategies for overcoming barriers. In addition, researchers 
questioned principals about R-Tech’ s sustainability during site visits to eight R-Tech districts in April 
2009. Principals’ responses are presented in the following sections. 

Barriers to Sustainability 

The spring 2009 survey asked principals to respond to a list of potential challenges to continuing R-Tech, 
indicating whether challenges were not a barrier, or were a minor, moderate, or substantial barrier to 
R-Tech’s sustainability. Results presented in Figure 7.1 indicate that nearly half of responding principals 
(48%) felt that “insufficient financial resources” was a moderate or substantial barrier to continuing 
R-Tech services. Smaller percentages of principals responded that student disinterest (29%), insufficient 
technology resources (27%), and lack of technical support (23%) were moderate or substantial banders to 
sustaining R-Tech. 
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■ Not a barrier nMinor barrier □ Moderate barrier □ Substantial barrier 


Figure 7.1. Principals’ perceptions of barriers to R-Tech sustainability. 

Source: R-Tech Principal/Facilitator Survey, spring 2009. 

Note. N=75. 

Overcoming Barriers 

The spring 2009 survey also asked principals about the methods by which their schools may address 
sustainability barriers. As presented in Table 7.9, more than half of responding principals (55%) indicated 
that R-Tech services would be incorporated into regular classroom instruction after grant funds expired, 
and 53% of principals indicated that they would seek additional funding to support R-Tech. A smaller 
percentage of principals (25%) planned to incorporate R-Tech services into district alternative education 
programs, and 11% indicated that they would discontinue the program when grant funds expire. 

Principals who responded “other” were provided with space to enter written comments, and four 
principals provided comments. Two principals wrote that they “do not know” how to sustain the program. 
Another indicated that district administrators would decide whether to continue R-Tech, and another 
wrote, “There is no solution to transportation problems and conflicts with extracurricular activities before 
and after school.” 
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Table 7.9 

Principals’ Strategies to Overcoming Barriers to Sustainability, as a Percentage of 
Respondents: Spring 2009 


Strategy 

Spring 2009 
(N=75) 

Incorporate R-Tech services into regular classroom instruction 

54.7% 

Seek additional funding 

53.3% 

Incorporate R-Tech services into an alternative education program 

25.3% 

Discontinue R-Tech services 

10.7% 

Other 

9.3% 


Source: R-Tech Principal/Facilitator Survey, spring 2009. 

Note. Percentages will not total to 100. Principals could respond to more than one category. 


Site visit interviews with principals in eight R-Tech districts provided more detailed information about 
the barriers principals confront in continuing R-Tech services once the grant expires. In interviews, 
several principals noted the financial challenge of sustaining the program. One district budgeted a 
substantial portion of grant funds to cover the salary of a full-time facilitator for its computer lab, and 
when funds expire, the position may be discontinued. The principal explained: 

[TEA] needs to know that grant funds are mainly used to pay that employee. If that [funding] 
stops, [the full-time facilitator] goes away. If [the full-time facilitator] goes away, there is no way 
[the services] can function... If [the facilitator] is not in [the lab] full-time, you can forget us 
doing this type of stuff. 

However, a principal in another district expressed substantially greater commitment to sustaining R-Tech. 
“This is a long-term project that will continue with local and state funding,” explained the principal. 
“Hopefully, more will be allocated at the state level. However we are committed to continuing the project 
with local and private funds if necessary.” 

Principals on several other campuses explained that if R-Tech did not positively affect students’ TAKS 
scores, they would not continue the program. “When it comes down to it, [the] objective is the TAKS 
test,” said one such principal, noting that R-Tech’ s future was dependent on its effect on TAKS outcomes. 
A principal on another campus expressed a similar view. “It has to do with the data,” explained the 
principal, “I am a numbers person. If a program isn’t making a difference in terms of [TAKS] results, I 
am not going to keep throwing money at it.” 

SUMMARY 

The Texas Legislature allocated about $6.3 million in R-Tech Cycle 1 funding to 63 rural districts, with 
individual district grants ranging from $16,000 to an established maximum of $200,000. In May 2009, 
Cycle 1 districts had used about half of total funding, and the amount of grant funds accessed by 
individual districts ranged from 0% to 96%. The largest share of funding (67%) was spent on supplies and 
materials in support of the grant, including purchases of technology hardware and software, furnishings 
for computer labs, tuition and textbooks for dual credit courses, and other instructional materials. Some 
districts budgeted purchases of technology hardware and software as capital outlay, which absorbed about 
10% of districts’ total first year purchases. Districts implementing self-paced programs and technology 
immersion programs, as well as R-Tech programs implemented as part of the regular school day spent 
more on supplies and materials to support implementation, which likely reflects substantial investments in 
technology resources. Districts spent less on professional and contracted services and salaries in support 
of R-Tech during the project’s first year (about 15% and 8% of all expended funds, respectively). 

Notably, districts implementing one-to-one tutoring programs with online instructional support spent 
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more on contracted services, largely because both districts implementing this type of program purchased 
R-Tech programming and services from a consortium that supports rural public schools. 

Districts that implemented R-Tech broadly, providing services to many students, experienced lower 
per-student costs than districts with narrower implementations targeted to specific student groups. As 
R-Tech expands to include more students during its second implementation year, many districts are 
expected to experience reductions in the per-student cost of implementation as program costs are spread 
across a greater number of students. In spite of expected reductions to the per-student cost of 
implementing R-Tech, nearly half (48%) of principals responding to the spring 2009 survey indicated that 
the lack of financial resources was a moderate or substantial barrier to sustaining R-Tech, and most 
(55%) indicated that R-Tech would be incorporated into regular classroom instruction when grant funds 
expire. 
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Chapter 8 

Summary of Findings and Discussion 


Recognizing that rural schools face challenges in providing supplemental instruction to support student 
achievement and are often limited in their ability to offer a diverse curriculum, the Texas Legislature 
authorized the R-Tech Pilot Program in 2007. R-Tech provides $8 million in funding to allow rural Texas 
districts to implement technology-based supplemental educational programs for students in Grades 6 
through 12 and to enable students attending rural high schools to participate in technology-based dual 
credit coursework and distance learning opportunities. R-Tech funding was awarded through a 
competitive grant process. Eligible districts must have enrolled fewer than 5,000 students and must not 
have been located in a metropolitan area in January 2007. Districts with limited course offerings and low 
accountability ratings received priority in grant funding. 

R-Tech funding was awarded in two grant cycles. R-Tech Cycle 1 grants awarded about $6.3 million in 
funding to 64 districts to be used during the May 1, 2008, through May 31, 2010, grant period. Cycle 2 
grants awarded about $1.5 million in grant funding to 19 districts to be used during the January 1, 2009, 
through May 31, 2010, grant period. 1 Grantee districts receive $200 per student per year in state funding 
for each student served by the grant and are required to provide $100 per student per year in matching 
funds. In funding R-Tech, the Legislature required that the program be evaluated to assess its 
effectiveness in improving student outcomes and specified that analyses consider how districts implement 
R-Tech, the program’s effects on teacher and student outcomes, and its cost effectiveness. To address 
these goals, the evaluation considers the following five research questions: 

1. How is R-Tech implemented across grantee districts and schools? 

2. What is the level of student participation in R-Tech? 

3. What is the effect of R-Tech on teachers? 

4. What is the effect of R-Tech on student outcomes? 

5. How cost effective is R-Tech? 

The evaluation will produce two interim reports (fall 2008 and winter 2010) and a final report (fall 2010). 
The first interim report (fall 2008) presented findings describing districts implementation plans (Research 
Question 1) drawn from analyses of districts’ grant applications to TEA. This report draws from a broader 
range of data sources to provide preliminary responses to each of the evaluation’s research questions. This 
chapter discusses findings relevant to each research question. Results are limited to districts that received 
Cycle 1 grant awards 2 and address R-Tech’ s first implementation year (May 2008-May 2009). 3 The 
chapter also discusses policy implications suggested by the report’ s analyses and concludes with an 
overview of the ongoing evaluation. 


'Three Cycle 2 districts also received Cycle 1 grant awards. 

“Although 64 districts initially received Cycle 1 grants, one district withdrew from the program without accessing 
grant funds and a second district deferred implementation of its R-Tech program to the grant’s second year, which 
reduces the number of districts included in most analyses to 62. The district deferring implementation of its R-Tech 
program to the second grant year accessed R-Tech funding in the grant’s first year and is included in analyses of 
R-Tech’s cost effectiveness (N=63). 

3 R-Tech’s first implementation year includes the 2008 summer session, as well as the fall 2008 and spring 2009 
semesters. 
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EVALUATION RESEARCH QUESTIONS 


The following responses to the evaluation’s research questions are preliminary and arc drawn from 
information collected during R-Tech’s first implementation year. Research questions are broad in scope 
and include sub-questions that address specific aspects of program implementation and effectiveness. The 
ongoing evaluation will incorporate information gathered during R-Tech’s second implementation year 
and will provide more complete responses to questions in the evaluation’s final report (fall 2010). 

Research Question 1 : How Is R-Tech Implemented Across Grantee Districts and 
Schools? 

The following sections address the characteristics of the districts and campuses receiving R-Tech Cycle 1 
grants, as well as the students they enroll, and consider the types of programs districts chose to 
implement, barriers to program implementation, and how banders were overcome. 

What are the characteristics of districts and schools that receive R-Tech grants? 

As previously noted, legislative requirements for R-Tech participation included a district enrollment of 
fewer than 5,000 students and a location outside metropolitan area. Districts with limited course offerings 
and high academic need, as demonstrated by 2007 accountability ratings, received priority in grant 
awards. 

R-Tech districts. Sixty-three districts participated in Cycle 1 of the R-Tech program. Districts were well- 
distributed throughout the state’s 20 ESC regions. On average, R-Tech districts enrolled 1,614 students, 
compared to the statewide average of 3,900 students. The proportion of R-Tech districts rated Exemplary 
or Recognized in 2007 lagged the state average. Although a majority of grantee districts (79%) received 
an Academically Acceptable rating, many implemented R-Tech to address an existing and, in some cases, 
growing achievement gap between low-income and more affluent students, as well as between minority 
and non-minority students. 

R-Tech schools. Of the 1 15 schools participating in R-Tech Cycle 1 grants, 51% were high schools, 40% 
were middle schools, 6% were elementary schools that served Grade 6, and 3% of schools served a range 
of grade levels (e.g., K through 12). On average, R-Tech campuses enrolled 408 students, compared to the 
statewide average of 617 students. In 2007, R-Tech campuses lagged the state average in the proportion 
of schools rated Exemplary and exceeded the state average in terms of schools rated Academically 
Unacceptable. 

What are the characteristics of students in R-Tech grantee districts and schools? 

Consistent with national enrollment trends in rural schools, R-Tech campuses enrolled a larger percentage 
of White students relative to statewide averages (63% vs. 37%) and smaller percentages of Hispanic (27% 
vs. 44%) and African American students (9% vs. 15%), and similar percentages of low income students 
(47% vs. 49%). In comparison to state averages, students attending R-Tech schools had slightly higher 
TAKS scores, lower dropout rates, and higher graduation and advanced course completion rates in 2007. 

What types of programs did R-Tech districts implement? 

Using descriptions of R-Tech programs included in district grant applications and progress reports to 
TEA, researchers identified five types of R-Tech programs implemented by Cycle 1 districts during 
R-Tech’s first year. Program types include: (1) self-paced instructional software providing remediation, 
tutoring, and credit-recovery; (2) dual credit and distance learning programs that allow students to earn 
college credit for courses that also fulfill high school requirements; (3) one-to-one tutoring with online 
instructional support; (4) technology immersion programs; and (5) iPods loaded with instructional 
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content. The program types are not discrete. That is, districts may operate more than one type of program. 
For example, some R-Tech districts offered dual credit instruction at the high school and implement a 
self-paced program focused on TAKS remediation at the middle school. 

In addition, researchers also categorized R-Tech programs as supplemental or non-supplemental 
depending on when students access R-Tech services. Supplemental programs provided R-Tech services 
outside of regularly scheduled classes (e.g., before or after school, during a free period) and non- 
supplemental programs integrated R-Tech services into regular class instruction. 

Self-paced instructional programs. Most Cycle 1 districts (87%) implemented some form of self-paced 
program during R-Tech’ s first year. Self-paced programs were generally delivered in computer labs and 
used software focused on providing students with targeted remediation and support. Programs often 
included diagnostic assessments that identified individual areas of weakness and provided students with 
lessons directed at specific academic skills. After completing units, students generally completed a post- 
assessment that measured learning and identified areas requiring further support. Some self-paced 
programs permitted students to complete entire courses of study (e.g., algebra) and recover credit for 
failed or incomplete coursework. Sixty percent of self-paced programs were implemented as 
supplemental instruction, where students accessed resources outside of regularly scheduled classes. 

Dual credit and distance learning programs. Less than a third (30%) of districts offered dual credit and 
distance learning opportunities as paid of R-Tech. Dual credit courses enable students in Grades 1 1 and 12 
to earn credit for college coursework taken while in high school. Such programs arc generally taught by 
university or community college instructors, and require that districts and college/university partners 
coordinate course schedules as well as attendance and grading requirements. R-Tech dual credit courses 
may be taken online or through video conferencing arrangements in which students submit work and take 
exams electronically. Surveyed students who participated in R-Tech dual credit programs reported high 
levels of satisfaction with their classes, indicating that dual credit courses were more challenging than 
their regular high school courses and were providing preparation for college. About 60% of districts 
offering dual credit courses implemented supplemental programs in which students participated in courses 
outside of regularly scheduled classes. 

One-to-one tutoring with online instructional support. Two R-Tech districts plan to implement 
programs in which tutors work with students one-on-one and students receive additional online support. 
Both districts contracted services from TxRED, a consortium that provides technology-based instructional 
support, including dual credit coursework, to rural Texas districts. Neither program served students during 
R-Tech’ s first implementation year, and it is not clear whether programs will be offered as supplemental 
or non-supplemental instruction. 

Technology immersion programs. Two districts implemented R-Tech as a technology immersion 
program in which all students and teachers received laptops. Teachers received training in the integration 
of technology in classroom instruction and used laptops to prepare and deliver instructional content. 
Students used laptops as paid of regular instruction and were able to take laptops home. Students with 
home Internet access were able to use laptops to access online resources at home. Researchers categorized 
both immersion programs as non-supplemental instruction because students primarily used laptops during 
regularly scheduled instruction. 

iPods loaded with instructional content. Two districts used R-Tech resources to purchase iPods and 
loaded machines with instructional resources that students accessed outside of school. One district 
distributed iPods loaded with core content area resources to all middle school students and another district 
provided iPods loaded with resources to support the development of English language skills to middle 
school ELL students. These programs were characterized as supplemental programs because students 
primarily used iPods outside of class. 
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What barriers limited the implementation of R-Tech programs? 


Data collected through site visit interviews and focus groups, as well as surveys of school staff and 
students participating in R-Tech indicate that the most substantial challenges to implementing R-Tech 
resulted from student resistance to the program and from transportation and scheduling conflicts that 
limited students’ ability to participate in services. Districts offering supplemental programs in which 
R-Tech services were offered after school experienced the greatest challenges. Respondents in such 
districts reported that some students simply would not stay after school. Conflicts with extra-curricular 
activities and students’ need to ride buses to get home also limited program participation. 

Principals and R-Tech facilitators also reported that technical issues created challenges to 
implementation. Some districts experienced problems resulting from outdated computer hardware and 
incompatible software, and others lacked the bandwidth to adequately support broad access to the 
Internet. In addition to technology challenges, districts experienced difficulties in communicating 
program goals to parents and teachers, and completing program reporting requirements. 

How were implementation barriers overcome? 

Districts used a range of strategies to increase student participation in R-Tech, including expanding 
R-Tech access times, providing incentives to encourage attendance (e.g., snacks), or making participation 
mandatory for some students. Some districts adjusted their implementation plans and incorporated 
R-Tech into the regular school day rather than offering services after school, and several districts added 
bus routes to accommodate students’ transportation needs. 

To address technology challenges, districts purchased new computer hardware and software, updated 
infrastructure, and worked closely with technical support staff to resolve issues. Principals and program 
facilitators also reported holding information sessions to communicate R-Tech’ s goals to staff and 
developing promotional materials to expand parents’ awareness of the program. In response to the spring 
survey, principals and program facilitators indicated that R-Tech implementation was facilitated by strong 
support from district administrators and teacher buy-in. Survey respondents highlighted the importance of 
grant funding, noting that the additional resources enabled districts to improve the quality of their 
technology resources and designate an R-Tech facilitator to support student and teacher access to R-Tech 
services. 

Research Question 2: What Is the Level of Student Participation in R-Tech? 

Student participation information reported through TEA’s data upload system indicates that access to 
R-Tech resources increased across the program’s first implementation year. About 1,400 students 
participated in R-Tech in summer 2008, about 8,800 participated in fall 2008, and nearly 12,800 
participated in spring 2009. The increase in student participation in R-Tech reflects variations in districts 
implementation schedules. While less than half of districts (47%) offered R-Tech services in summer, 
nearly all districts (92%) had implemented the program in spring 2009. The sections that follow provide 
information about students’ level of participation in R-Tech services, including how students were 
identified for services, the characteristics of students who received services, and the amount of time 
students spent using R-Tech resources. 

How are students identified for R-Tech services? 

As discussed earlier in this chapter, districts varied in how they implemented R-Tech and differences in 
program configurations affected how students were identified to receive services. Several districts 
implemented school-wide technology immersion programs in which all students used R-Tech resources. 
Other districts targeted their programs to a specific subject area (e.g., ELA or math), and students 
accessed resources as paid of class time scheduled in computer labs. In other districts, R-Tech services 
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were offered outside of regular class time to students who were “identified” for services. Results from the 
surveys of and site visit interviews with principals and R-Tech facilitators indicate that most students 
were identified for R-Tech because of weak academic outcomes, including poor TAKS scores, failing 
grades, teacher referrals, and prior failures. Some districts also referred students with poor attendance as a 
means to help students “catch up.” Several districts targeted R-Tech services to specific student groups. 
For example, one district provided R-Tech services designed to help ELL students build language skills 
and another district implemented a behavior modification program for students receiving disciplinary 
referrals. 

What are the characteristics of students who participate in R-Tech services? 

Comparisons of the demographic characteristics of students who received R-Tech services to those who 
did not revealed few differences for the fall 2008 or spring 2009 reporting periods. That is, the 
characteristics of students who participated in R-Tech during the school year reflected the larger school 
population in terms of their ethnicity and economic characteristics. Flowever, students who participated in 
R-Tech as part of the 2008 summer session differed from the population of students who did not receive 
such services. R-Tech summer school students were more likely to be African American (18% vs. 9%) 
and from low income backgrounds (55% vs. 46%) than students who did not participate in R-Tech during 
summer school. Perhaps most notable is that students who participated in R-Tech as part of summer 
school were more likely to be in middle school than in high school (65% vs. 35%), and the largest 
proportion of R-Tech summer school students were in eighth grade (29%). This finding may suggest that 
districts used R-Tech to provide remediation to middle school students in order to reduce retention rates 
and support students’ transition to high school. 

How many hours per week do students receive R-Tech services? 

Districts also indicated the average number of hours each participating student received R-Tech services 
each week in their reports to TEA for each of the three grant periods. Analysis of student participation 
data indicate that R-Tech services offered as part of summer school programs were more intensive than 
services offered during the school year - . Students participating in R-Tech as part of the 2008 summer 
session accessed resources for an average of 8.5 hours per week. Flowever, in fall 2008 students accessed 
resources for an average of 3.7 hours per week, and in spring 2009, students averaged 3.8 hours per week 
using R-Tech resources. The intensity of R-Tech services offered during the summer session may reflect 
the condensed timeline for summer courses, but it also may indicate that students identified for R-Tech 
summer programs spent more time with resources in order to make up coursework or recover the credits 
needed to move to the next grade level in the fall. 

Research Question 3: What is the Effect of R-Tech on Teachers? 

Technology has the potential to improve rural teachers’ access to professional development through the 
provision of online training opportunities, and several states have implemented statewide programs 
focused on providing rural teachers with technology-based training to support and improve instruction. 
R-Tech facilitates improved teacher performance in rural districts by enabling districts to use grant funds 
to provide teachers with professional development in research-based instruction and strategics to increase 
course rigor. The following sections provide information about the effect of R-Tech on teachers, focusing 
on how districts implement R-Tech training, the types of training offered, and the effects of training on 
teachers’ classroom instruction. 
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How do grantee districts and schools implement the teacher training component of the 
R-Tech program? 

Although all Cycle 1 grantee districts reported plans to offer professional development in support of 
R-Tech in their grant applications, results from surveys and focus group interviews with teachers on 
R-Tech campuses indicate that R-Tech professional development was weakly implemented during the 
program’s first year. Variations in districts implementation strategics and poor communication of grant 
goals affected teachers’ access to training opportunities. Across site visit districts, principals expressed 
the need to improve communication about the grant, its goals and expectations, and the opportunities it 
provides for teachers in terms of instructional resources and training. 

Results from teacher surveys conducted in fall 2008 and spring 2009 indicate that few teachers 
participated in R-Tech training during the grant’s first year. Less than 5% of teachers responding to the 
fall 2008 survey (54 individuals) and 38% of spring survey respondents (215 individuals) knew they had 
participated in R-Tech professional development. However, some survey respondents were unsure if the 
professional development they received was related to R-Tech or not. 

What types of training do teachers participate in as part of the R-Tech program? 

Of the 215 teachers responding to the spring survey who participated in R-Tech professional 
development, most received training in TAKS/TEKS preparation, technology-based instruction, working 
with at-risk students, using instructional hardware and software, and aligning curriculum. Most 
professional development activities were presented in face-to-face formats, and few surveyed teachers 
reported accessing online training opportunities provided through R-Tech. 

Although few surveyed teachers reported participating in online professional development activities, 
district-provided teacher usage data indicates that about 800 teachers (about 22% of all teachers on 
R-Tech campuses) received online training as part of R-Tech. Across the 2008-09 school year, teachers 
who participated in online training did so for an average of 16 hours. Middle school teachers had higher 
levels of participation (approximately 19 hours, on average) than high school teachers (approximately 16 
hours, on average), and both middle and high school teachers had higher levels of participation in fall 
2008 than in spring 2009. 4 

What is the effect of R-Tech teacher training on teacher effectiveness? 

Results of the spring 2009 teacher survey suggest that R-Tech professional development activities had 
little effect on teachers’ classroom practices during the pilot’s first year - . Less than half of surveyed 
teachers agreed that R-Tech had affected classroom instruction or lesson planning, and less than a third 
were using students’ PEPs to inform instruction. However, the effects of training may differ across 
districts implementing different types of R-Tech programs. During focus groups conducted as paid of 
spring site visits, teachers working at a school implementing a technology immersion program reported 
receiving training in how to integrate technology into instruction. While immersion teachers were 
overwhelmed by training content, they were also enthusiastic about the potential of technology to 
transform classroom instruction. Focus group teachers on campuses implementing self-paced instructional 
programs said they appreciated learning about vendor-provided software, but did not link training content 
to instructional practice. 

While few teachers reported improvements in instructional practice from R-Tech training, focus group 
teachers reported other positive effects from the grant. Teachers said R-Tech increased their access to 


4 No teachers participated in training in summer 2008. 
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technology resources and their ability to differentiate instruction. In addition, focus group teachers felt 
that students were more engaged when they used technology-based resources. 

Research Question 4: What Is The Effect of R-Tech on Student Outcomes? 

Although the ongoing evaluation will include a broader range of student outcome indicators, the findings 
presented in this report are limited to R-Tech’ s effects on students’ TAKS scores, which as noted in 
chapter 1 , may produce misleading conclusions about the program’ s effectiveness. Recognizing this 
limitation, readers are asked to interpret the findings presented in the following sections with caution. The 
final evaluation report (fall 2010) will provide more comprehensive analyses of R-Tech’s effects on 
student outcomes for the grant’ s full implementation period. 

Is there a relationship between students’ access time and TAKS outcomes? 

Increased hours spent participating in R-Tech services had no effect on students’ 2009 TAKS scores. 
Results may be due to the limited time most students spent in R-Tech activities — more than 80% of 
students receiving R-Tech services in fall 2008 and spring 2009 averaged 4 or fewer hours per week using 
resources — and TAKS, as well as other standardized achievement tests, may lack the sensitivity to 
measure subtle achievement gains. 

Is there a relationship between R-Tech program configurations and students’ TAKS 
outcomes? 

The following sections consider whether the manner in which districts implemented R-Tech may have 
affected students’ TAKS outcomes. Findings consider whether students who participated in R-Tech as 
part of the regular school day (i.e., non-supplementary programs) experienced different outcomes from 
students who participated in supplementary services offered outside of classroom instruction (e.g., before 
or after school) and whether differences in the types of R-Tech programs affects TAKS outcomes. 
Although researchers identified five types of R-Tech programs (i.e., self-paced instruction, dual 
credit/distance learning programs, one-to-one tutoring with online instructional support, technology 
immersion programs, and iPods loaded with instructional content), the small number of districts offering 
one-to-one tutoring with online instructional support, technology immersion programs, and iPods loaded 
with instructional content precluded the inclusion of these program types in analyses. 

Supplementary vs. non-supplementary programs. Implementing R-Tech as a supplementary program 
offered outside of regularly scheduled classes demonstrated a consistently negative relationship with 
students’ TAKS outcomes across testing areas; however, with exception social studies, the relationships 
were not statistically significant. Although preliminary, these results may indicate that offering R-Tech 
services during regular instructional hours or during elective periods may produce greater TAKS gains 
than offering services outside regular instructional hours. 

Program type. R-Tech programs that used self-paced computer software had a statistically significant 
negative relationship with students’ TAKS reading/ELA scores, but no significant relationship with 
students’ TAKS mathematics, science, and social studies scores. R-Tech dual credit and distance learning 
programs did not demonstrate a statistically significant relationship with students’ TAKS reading/ELA 
and mathematics scores. These results should be interpreted with caution because it was not possible for 
researchers to control for the many factors that affect students’ testing outcomes or for the non-random 
identification of students for R-Tech services. 

Research Question 5: How Cost Effective Is R-Tech? 

While student outcomes arc an important indicator of the effectiveness of an educational intervention, 
policymakers are also concerned with whether the funds used to implement a program are used 
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effectively. Cost-effectiveness analyses generally compare multiple programs designed to achieve the 
same outcome; however, in the case of this evaluation, the only program under consideration is R-Tech. 
Because researchers did not have access to data on similar programs, the cost-effectiveness analysis 
presented in this report compares costs across different approaches to implementing R-Tech, using five 
researcher-identified program configurations discussed earlier in this chapter and considers cost 
differences between districts implementing R-Tech as a supplemental vs. non-supplemental program (i.e., 
as part of regular instruction). Recall that program configuration categories are not discrete, and districts 
may be included in more than one category. 

The cost-effectiveness analysis seeks to understand how districts allocated funding across R-Tech’ s first 
year, which program configurations made the most effective use of funding, and whether R-Tech will be 
sustained when grant funding expires. The findings presented in this section are limited to state grant 
funding and do not consider districts' use of matching funds. Further, findings rely on districts’ use of 
grant funds during the R-Tech’ s first grant year, in which many districts invested heavily in technology 
hardware, software, and infrastructure. Although these expenditures are included in analyses, their per- 
student costs are overstated because the technology resources will be used by many more students during 
the grant’s second year and the years that follow. Findings presented in the evaluation’s final report (fall 
2010) will offset this limitation to some extent by measuring per-student costs in terms of all R-Tech 
students served by resources for the full 2-year grant period. 

How are grant funds allocated in R-Tech districts? 

In order to understand how R-Tech districts allocated grant funds across the project’s first year, 
researchers examined district expenditure data submitted to TEA through May 2009 using the Agency’s 
ER system. The ER system includes five expenditure categories: (1) payroll costs, (2) professional and 
contracted services, (3) supplies and materials, (4) other operating costs, and (5) capital outlay. 
Researchers also analyzed district budgets included in grant applications to identify the puiposes of the 
costs recorded in each expenditure category. 

Payroll costs. Funds allocated to payroll costs must be spent on district employees or non-employees 
(e.g., consultants) who work part-time or full-time on R-Tech activities. Only 8% of the funding 
expended on R-Tech through May 2009 had been allocated to payroll costs. In grant applications, districts 
indicated that payroll expenditures would be used to cover the salaries of R-Tech facilitators, computer 
lab staff, extra duty pay for teachers who worked before or after school to provide R-Tech services, and 
for substitutes to enable teachers to participate in professional development. Districts that implemented 
dual credit and distance learning programs tended to have higher average expenditures on payroll costs 
than R-Tech districts generally ($4,200 vs. $3,603, on average), while districts that used R-Tech funds to 
purchase iPods with educational programs targeted to specific student populations (e.g., ELL) expended 
well below the average amount spent on payroll costs for all R-Tech districts ($425 vs. $3,603). The 
reasons for these differences are not clear. 

Professional and contracted services. In May 2009, about 15% of all expended state grant funding was 
spent on professional and contracted services. District grant applications indicated that funding was used 
to pay for educational software, registration fees and tuition for dual credit or distance learning courses, 
technical support, and professional development activities. Districts implementing dual credit coursework 
and distance learning programs tended to have higher average expenditures in this category ($12,065) 
than the average for all R-Tech districts ($6,648). Perhaps most noteworthy, however, is the amount 
expended for professional and contracted services by districts offering one-to-one tutoring and online 
instructional support ($52,715). Both districts in this category contracted with TxRED to provide dual 
credit programs at the high school and technology-based instructional support at the middle school. 
Districts allocated an average of $81,130 for TxRED services and accessed $52,715 of this funding 
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during R-Tech’s first year, although neither district had implemented the program for students in May 
2009. 


Supplies and materials. Across all R-Tech districts, the largest share of state-funded grant revenue was 
used to purchase supplies and materials (67%) during the pilot’s first year. In grant applications, districts 
indicated that grant funds would be used to purchase computer hardware and software, furnishings for 
computer labs, LCD projectors, printers, textbooks for dual credit courses, and other materials related to 
implementing a technology-based educational program. Districts implementing self-paced programs 
focused on remediation, tutoring, and credit recovery; technology immersion programs; and R-Tech 
programs implemented as part of the regular school day (i.e., non- supplemental programs) tended to 
spend more on supplies and materials than districts implementing supplemental programs offered outside 
of regularly scheduled classes. This finding reflects the substantial investment in technology resources 
that such districts required in order to implement their R-Tech programs. 

Other operating costs. Districts did not report spending any grant revenue on other operating costs 
during R-Tech’s first year, although several districts budgeted for other operating costs, including travel 
costs for staff and students participating in R-Tech, as well as printing costs, in grant applications. 

Capital outlay. Districts may use capital outlay funds to purchase tangible, nonexpendable property that 
will be used for more than 1 year. Across districts, about 10% of expended R-Tech funds were spent for 
capital outlay, and grant applications indicated that funding was used to purchase technology resources, 
including computer hardware and software. Similar to results for “supplies and materials,” districts that 
implemented self-paced programs and districts that included R-Tech as paid of the regular school day 
tended to spend more average funding on capital outlay. 

Which R-Tech program configurations make the most effective use of funding as 
measured by reduced program costs? 

In spite of substantial start-up investments related to the purchase of technology resources, districts that 
implemented R-Tech for larger numbers of students experienced reduced per-student costs. Across all 
R-Tech districts, the average cost per student of implementing the program in terms of state funding was 
about $420 during the program’ s first year. Note that this amount overstates the average cost per student 
because districts invested substantial R-Tech funding in technology resources that will be used over the 
life of the grant and beyond. As more students access R-Tech services during the grant’s second year, per- 
student costs are expected to drop. The following sections discuss how districts’ implementation 
strategies affected costs during R-Tech’s first year, and consider program size, the type of program 
implemented, and whether R-Tech services were implemented as supplemental or non-supplemental 
programs. Findings are preliminary and are not linked to student outcomes; therefore, it is not possible to 
determine how effectively funds were spent at this time. 

Program Size 

Not surprisingly, districts that served large numbers of students in R-Tech services had the lowest per- 
student program costs over the pilot’s first year. Districts that served 500 or more students had average 
per-student program costs of about $111, while districts that served fewer than 50 students had average 
per-student program costs of about $1,521. Mid-sized programs, serving between 100 and 249 students, 
had average per-student costs of about $244. 
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Type of R-Tech Program 

Self- paced instructional programs. Most R-Tech districts (87%) offered some form of self-paced 
instructional program, including online tutoring, remediation, and credit recovery. As noted in the 
previous section, this program configuration required substantial investment in technology resources to 
expand and update existing computer labs and, in some cases, to purchase laptop computers that students 
could use at home. Districts implementing self-paced programs served, on average, about 256 students in 
R-Tech and incurred average per-student costs of about $428 during the program’s first year. These costs 
are expected to drop in the grant’ s second year as more students are served with the technology resources 
purchased through R-Tech. 

Dual credit and distance learning programs. About 30% of districts offered dual credit and distance 
learning programs using R-Tech funding. These districts spent less on technology purchases, but more on 
contracted services that provide technology-based instruction. Overall, districts implementing this type of 
program served an average of 177 students and experienced the lowest per-student costs — $198 per 
student, on average — in R-Tech’ s first year. However, unlike investments in technology, revenue spent on 
contracted services will not extend beyond the contract period, so it is not clear whether districts 
implementing dual credit and distance learning programs will experience a reduction in per-student costs 
during the project’s second year. 

One-to-one tutoring and online instructional support. Neither of the two districts offering this type of 
program served students during R-Tech’ s first year; therefore, it is not possible to calculate a per-student 
cost of implementation. 

Technology immersion projects. The two districts that implemented R-Tech as a technology immersion 
project, in which each student was provided with a laptop and access to technology resources, invested 
heavily in technology resources during the program’s first year, spending an average of $67,278 on 
supplies and materials in support of the grant. These districts implemented R-Tech broadly, serving 306 
students, on average, which reduced their per-student costs of implementation relative to other R-Tech 
districts. Technology immersion districts had an average per-student program cost of about $269, while 
the average for all R-Tech districts was $420 across the first year of implementation. Similar to self-paced 
programs, technology immersion projects may expect their per-student costs to decline in the R-Tech’ s 
second year as more students access technology resources purchased with grant funds. 

iPods with targeted instructional resources. Two districts used R-Tech funding to purchase iPods 
loaded with instructional resources for a targeted group of students (e.g., ELLs). These districts 
implemented the program narrowly, serving only 55 students, on average, and had average per-student 
costs of about $358. 

Supplemental vs. Non-Supplemental Services 

Although R-Tech was intended to provide supplemental instructional support offered outside of regular 
instruction (e.g., before or after school), many districts (40%) implemented R-Tech as paid of daily 
instruction. Some districts implemented technology immersion projects in which students used laptops in 
core classes, some districts purchased sets of laptops that teachers used in the classroom, and other 
districts expanded access to computer labs, in which teachers scheduled class time for students to access 
services. Comparisons of per-student costs across the two approaches to implementation indicate that 
districts that incorporated R-Tech into regular instruction (i.e., non-supplemental programs) experienced 
substantially lower per-student costs relative to districts that implemented supplemental programs ($187 
vs. $612). This difference results from the number of students who were able to access R-Tech services. 
Districts offering supplemental programs served an average of 172 students, while districts implementing 
R-Tech as paid of the regular school day served an average of 350 students across the program’s first year. 
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Is R-Tech sustainable? 


Information about districts’ ability to sustain R-Tech services once grant funds expire was gathered from 
spring 2009 surveys of principals on R-Tech campuses and through principal interviews conducted during 
spring site visits to eight R-Tech districts. Principals’ survey and interview responses reflect ambivalence 
about the sustainability of R-Tech. At the end of R-Tech’s first year, principals were unsure of R-Tech’s 
effects on testing outcomes, and whether they would continue to fund the program when the grant 
expired. According to surveyed principals, lack of funding to sustain services is the greatest barrier to 
extending R-Tech beyond the grant period. Approximately half (48%) of responding principals indicated 
that insufficient financial resources created a moderate or substantial barrier to continuing the program. 
Surveyed principals also identified strategics to continuing R-Tech, and most principals (55%) indicated 
that they would incorporate R-Tech into regular classroom instruction rather than providing supplemental 
instruction offered outside of scheduled classes. A somewhat smaller percentage of principals indicated 
that they would seek additional funding sources (53%), and 25% planned to incorporate R-Tech into an 
alternative education program for at-risk students. 5 During site visit interviews, principals underscored the 
financial challenges to continuing R-Tech, but also emphasized concerns about the program’s effect on 
TAKS scores. Some interviewed principals said that if R-Tech was not effective in improving students’ 
TAKS outcomes, they would not continue the program. 

DISCUSSION OF FINDINGS 

The following sections discuss the second interim report’s key findings and provide recommendations to 
policymakers and individuals charged with implementing R-Tech. Recommendations strive to inform 
subsequent policy decisions and to shape implementation strategies for the grant’s second year. 

Supplemental vs. Non-Supplemental Implementation 

While R-Tech is intended to provide supplemental instruction offered outside of students’ regularly 
scheduled classes, many districts planned R-Tech as paid of regular class instruction, designing programs 
that incorporated laptops in daily lessons or expanding access to computer labs and updating equipment to 
enable teachers to schedule class time for students to use R-Tech resources. In addition, some districts 
that initially implemented R-Tech as a supplemental program encountered challenges that limited the 
program’s effectiveness when offered outside of regularly scheduled classes. In some districts, 
transportation challenges created banders to students’ program participation when bus schedules 
prevented students from arriving early or staying after school. Some students were unable to participate in 
R-Tech services because of conflicts with extra-curricular activities, and others simply would not 
participate in instruction offered outside the regular school day. In response to these challenges, some 
districts adjusted their implementation plans to incorporate R-Tech into the school day and encouraged 
teachers to permit students to visit computer labs during class time. 

Although preliminary, the analyses presented in this report suggest that districts that implemented R-Tech 
as part of regular - instruction experienced improved outcomes relative to districts that adhered to the 
program’s intent and implemented supplemental programs. Early results from analyses of R-Tech’s 
effects on students’ TAKS scores indicate that students who participated in R-Tech as part of regular 
instruction (i.e., non-supplemental programs) may enjoy improved testing outcomes relative to students 
participating in supplemental programs. Results from site visits suggest that teachers who used resources 
as part of classroom instruction provided greater direction for student learning with R-Tech resources and 
used program-provided data to a greater extent to individualize technology-based lessons and monitor 
student progress. Further, preliminary findings from the analysis of R-Tech’s cost effectiveness indicate 
that districts that implemented non-supplemental programs experienced lower per-student program costs, 


Principals were able to indicate multiple strategies for sustaining R-Tech; percentages will not total to 100. 
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in large part because greater numbers of students were able to access resources when services were 
included as part of regular instruction. These results suggest that districts that implement R-Tech as paid 
of regular instruction may be using resources more effectively and efficiently than districts that offer 
supplemental programs. 

Given these findings, policymakers may want to consider whether program guidelines should be revised 
to encourage districts to implement R-Tech services as paid of classroom instruction, as well as offering 
supplementary programs. Expanding R-Tech to regular instruction would enable greater student access to 
resources and increase teacher awareness of the program and its instructional benefits. Further, the 
expansion of R-Tech to regular classroom instruction would reduce the per-student cost of the program, 
and it is possible that expanded access to resources will produce positive effects on student achievement 
outcomes. 

Communication About R-Tech 

Poor communication of R-Tech’ s goals and expectations created implementation challenges across 
R-Tech districts. Within districts, communication break downs tended to occur between the 
organizational tiers that structure employment in public schools. For example, in many districts, central 
office staff charged with overseeing grant application processes did not communicate R-Tech goals and 
expectations to individuals at the campus-level who were responsible for implementing the program, 
while at the school level, some administrators did not provide sufficient information to teachers about 
R-Tech. Administrators in R-Tech districts also felt their programs could be improved if they had 
opportunities to communicate with staff in other grantee districts about implementation strategies. 

Within district communication. In many districts, campus-level staff did not participate in grant 
application processes and were unaware of grant requirements. In particular, few R-Tech administrators 
and facilitators knew that the grant required the development of PEPs to monitor student progress, and at 
least one district misunderstood R-Tech’ s access requirements. Administrators in this district understood 
that access to resources was to be provided for 10 hours a day rather than 10 hours a week. The district 
used grant funding to pay for a full-time facilitator for its computer lab, and to provide extra-duty pay for 
teachers who staffed the lab before and after school. While teachers in the district appreciated students’ 
expanded access to computers, school administrators noted that the facilitator’s position and access to the 
lab would be discontinued when grant funds expired. The poor communication of R-Tech requirements 
suggests that district-level staff who complete grant applications may want to spend time ensuring that 
campus-level staff charged with implementation have complete information about program goals and 
requirements. Districts may also want to ensure that future grant writing activities engage campus-level 
staff, including teachers, in grant application and planning activities. 

Further, poor communication of project goals between campus-level administrators and teachers also 
created implementation barriers. In many districts, teachers were unaware of R-Tech resources and 
training opportunities. Campus-level administrators arc advised to place greater emphasis on 
communicating grant goals and encouraging teachers’ use of resources and to include teachers in 
implementation activities. 

Between district communication. Several campus-level administrators and R-Tech facilitators in site 
visit districts expressed a desire to exchange information with staff in other R-Tech districts, and asked 
that TEA facilitate opportunities for districts to communicate with one another. Should TEA choose to 
facilitate such activities during the project’s second year, many implementation challenges may be 
overcome through shared information about successful implementation strategies and improved program 
outcomes. 
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THE ONGOING EVALUATION 


The final evaluation report (fall 2010) will present results for the full R-Tech implementation period (May 
2008-May 2010) and will include more conclusive findings about the program’s effects on students and 
teachers, and how program configurations may affect student achievement outcomes and project costs. 
Many of the limitations of this report, including incomplete data on student achievement outcomes, and 
uneven first year implementation of R-Tech across districts, will be offset by the inclusion of information 
collected during R-Tech’ s second implementation year in the final report. 

During R-Tech’ s second implementation year, the evaluation’s ongoing activities will include spring 
2010 surveys of principals and teachers on R-Tech campuses, R-Tech facilitators, and students receiving 
R-Tech services during the program’s second year. Surveys administered at the conclusion of the R-Tech 
grant period will provide information about fully implemented programs, ongoing challenges to 
implementation, as well as how challenges may have been overcome. Surveys will also identify changes 
in implementation roles and stakeholders’ perceptions of the program’s benefits. In addition, the spring 
2010 surveys will ask principals and R-Tech facilitators to identify which of the researcher-identified 
program categories used in the evaluation best describes their districts’ implementation of R-Tech. The 
inclusion of program categories on the surveys will allow researchers to refine program descriptions and 
more fully assess the effects of program configurations. Researchers will visit R-Tech districts again in 
spring 2010. Site visits will include interviews with principals and R-Tech facilitators, focus group 
discussions with teachers and students, and observations of R-Tech service delivery. Site visits will 
identify how programs may have changed during R-Tech’ s second implementation year and how changes 
may have shaped program outcomes. 

In addition to surveys and site visits, the ongoing evaluation will include a broader range of student 
outcome measures, including course completion data, graduation rates, and indicators of college 
readiness, as well as TAKS scores. The expanded student outcome data will provide a more complete 
view of R-Tech’ s effects on student achievement outcomes, including whether students are able to 
recover the credits needed to graduate on time. The final evaluation report will consider how different 
program configurations may affect student outcomes. In addition, the ongoing evaluation will incorporate 
information on districts’ full use of state grant funding, which will enable a more accurate assessment of 
R-Tech’ s cost effectiveness. 
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Appendix A 

The Online Principal and R-Tech Facilitator Survey 


The evaluation included information gathered through voluntary, online surveys of principals of R-Tech 
campuses and R-Tech program facilitators administered in fall 2008 and spring 2009. The surveys asked 
principals and facilitators a common set of questions about R-Tech’ s implementation, the program’s 
effects on students and teachers, and campus-level goals for R-Tech. Principals were routed to a set of 
open-ended questions asking about their goals for R-Tech’ s effects on teachers, as well as how they will 
know if these goals have been met. R-Tech facilitators were routed to a set of questions asking about the 
training they may have received to support the implementation of R-Tech. Principals who also acted as 
R-Tech facilitators responded to both sets of questions. The spring 2009 survey also included items 
asking about principals’ and facilitators’ roles in implementation, and how challenges to implementation 
may have been overcome during R-Tech’ s first year. The spring survey also asked principals about the 
sustainability of R-Tech after grant funds expire. 

This appendix contains information about survey administration procedures, response rates, and the 
characteristics of survey respondents. It also contains supplemental tables that present additional 
information cited in report chapters and copies of the fall and spring principal and R-Tech facilitator 
surveys. 

SURVEY ADMINISTRATION 
Fall 2008 

In November 2008, the principal of each of the 115 campuses that districts indicated were implementing 
R-Tech in their grant applications was sent a link to a voluntary, online survey of principals and R-Tech 
facilitators. Principals were asked to forward the link to the individual or individuals who acted as R-Tech 
facilitators on their campuses. Principals and facilitators were provided with 3 weeks to complete the fall 
2008 survey and received multiple reminders about survey deadlines. Given weak survey response rates 
at the survey’s deadline, evaluators kept the survey open for 3 additional weeks and sent additional 
reminders encouraging principals and facilitators to participate. 

Spring 2009 

Identical procedures were used to administer the principal and facilitator survey in May 2009. However, 
survey links were sent to the principals at the revised list of 115 campuses that R-Tech districts indicated 
were implementing the program in the spring 2008. Similar to the fall survey, weak response rates at the 
close of the spring survey caused evaluators to keep the survey open beyond its deadline, but due to the 
close of the school year and project timelines, the survey was extended only for an additional 10 days. 

The shortened survey timeline is likely the cause of reduced response rates for the spring 2009 
administration. 

Number of Survey Respondents and Response Rates: Fall 2008 and Spring 2009 

Number of survey respondents. In fall 2008, 153 individuals responded to the online survey of 
principals and R-Tech facilitators. Of the respondents, 66 acted only as a principal on an R-Tech campus, 
71 acted as R-Tech facilitators and were not principals, and 16 served in both roles. Twelve campuses 
responding to the survey had multiple R-Tech facilitators and had multiple facilitator responses to the 
survey (11 campuses had two facilitator survey respondents and one campus had 1 1 facilitator survey 
respondents). 
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In spring 2009, 136 individuals responded to the online survey. Of spring 2009 respondents, 60 
individuals acted only as a principal, 61 individuals acted as a facilitator and were not principals, and 15 
individuals served in both roles. Again, seven campuses had more than one individual acting as the 
R-Tech facilitator (four campuses had two facilitator respondents, two campuses had three facilitator 
respondents, and one campus had nine facilitator respondents). 


Table A.l 

The Number of Responses, by Respondent Type: Fall 2008 and Spring 2009 


Respondent Type 

Fall 2008 

Number of Responses 

Spring 2009 
Number of Responses 

Principal only 

66 

60 

R-Tech facilitator only 

71 

61 

Principal and facilitator 2 

16 

15 

Total 

153 

136 


Sources: R-Tech Principal/Facilitator Survey, fall 2008 and spring 2009 
“Respondent serves in both roles. 


Response rates are calculated at the campus-level for both principals and R-Tech facilitators. Because 
each campus may be reasonably understood to have one principal, the campus-level response rate for 
principals may be understood as the principal response rate for the survey. However, for R-Tech 
facilitators, the same reasoning does not apply. R-Tech requires that grantee districts provide an R-Tech 
facilitator to support implementation, monitor student PEPs, and monitor technology access and use. In 
addition, districts are required to have on-site, or campus-level, R-Tech facilitators to support students’ 
daily use of R-Tech resources and report student usage (TEA, 2008b). In most districts in which R-Tech 
facilitators responded to the survey, both sets of responsibilities were managed by campus-level 
facilitators, and, as noted earlier, some schools had more than one campus-level facilitator. Further, some 
respondents indicated that they acted as the facilitator for the district, but in most cases, these individuals 
were located on a campus (e.g., high school principal or middle school technology coordinator). Because 
districts varied widely in the ways in which they assigned facilitator responsibilities and because it is not 
clear how many individuals acted as facilitators across respondent and non-respondent campuses, it is not 
possible to establish facilitator-level response rates for the surveys. 

Campus-level response rates. Campus-level response rates are calculated for the total number of 
campuses participating in R-Tech in fall 2008 and spring 2009 (N=l 15 and N= I 1 5, respectively) and 
whether each R-Tech campus had a principal or facilitator who responded to the survey. Campuses with 
multiple facilitators responding to the survey are counted only once in the calculation of response rates. 
Principals who served dual roles — principal and facilitator — are included in the calculation of response 
rates for both principals and facilitators. 
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Table A.2 

Campus-Level Response Rates, Principal and R-Tech Facilitator: Fall 2008 and 
Spring 2009 



Fall 2008 

Spring 2009 


(N= 

115) 

(N=115) 


Campuses 
with Survey 
Respondents 

Percentage of 
Campuses 
Responding 

Campuses with 
Survey 
Respondents 

Percentage of 
Campuses 
Responding 

Principal 

82 

71.3% 

75 

65.2% 

R-Tech Facilitator 

75 

65.2% 

65 

56.5% 


Sources: R-Tech Principal/Facilitator Survey, fall 2008, spring 2009. 

Notes. Campuses with multiple facilitators responding to the survey are counted only once in the 
calculation of response rates. Principals who serve dual roles — principal and facilitator — are 
included in the response rate calculations for both principals and facilitators. 

CHARACTERISTICS OF SURVEY RESPONDENTS 

Table A.3 presents the characteristics of respondents to the fall 2008 and spring 2009 principal and 
R-Tech facilitator surveys. 


Table A.3 

The Characteristics of Principal/Facilitator Survey Respondents, as a Percentage of Respondents, 
by Role: Fall 2008 and Spring 2009 


Characteristic 


Fall 2008 



Spring 2009 

Principals 

(n=82) 

Facilitators 

(n=71) 

All 

Respondents 

(N=153) 

Principals 

(n=75) 

Facilitators 

(n=61) 

All 

Respondents 

(N=136) 

Gender 

Male 

65.9% 

23.9% 

46.4% 

70.7% 

19.7% 

47.8% 

Female 

34.1% 

76.1% 

53.6% 

29.3% 

80.3% 

52.2% 

School level 

Middle school 

36.6% 

15.5% 

26.8% 

40.0% 

24.6% 

33.1% 

Fligh school 

58.5% 

67.6% 

62.7% 

50.7% 

52.5% 

51.5% 

Other 3 

4.9% 

16.9% 

10.5% 

9.3% 

23.0% 

15.4% 

Experience (average years) 

At current school 

6.6 

11.3 

OO 
O O 

6.8 

8.1 

7.4 

In current position 







at current school 

3.3 

5.8 

4.4 

3.7 

4.7 

4.1 


Sources'. R-Tech Principal/Facilitator Survey, fall 2008, spring 2009. 

a “Other” types of R-Tech schools included combined middle and high schools, district level school, and an 
intermediate school. 
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SUPPLEMENTAL TABLES 

This section presents supplemental information referenced in report chapters. 


Table A.4 

Principals’ and Facilitators’ Roles in R-Tech Planning and Implementation, as 
a Percentage of Respondents by Task: Fall 2008 and Spring 2009 



Principals 

Facilitators 

Role 

(N=75) 

(N=61) 

Planning implementation 

No involvement 

12.0% 

19.7% 

Minor involvement 

18.7% 

14.8% 

Moderate involvement 

37.3% 

16.4% 

Substantial involvement 

32.0% 

49.2% 

Identifying students 

No involvement 

14.7% 

24.6% 

Minor involvement 

18.7% 

16.4% 

Moderate involvement 

34.7% 

19.7% 

Substantial involvement 

32.0% 

39.3% 

Monitoring students’ use 

No involvement 

8.0% 

13.1% 

Minor involvement 

33.3% 

16.4% 

Moderate involvement 

49.3% 

31.1% 

Substantial involvement 

9.3% 

39.3% 

Communicating with parents 

No involvement 

10.7% 

23.0% 

Minor involvement 

33.3% 

32.8% 

Moderate involvement 

36.0% 

26.2% 

Substantial involvement 

20.0% 

18.0% 

Participating in training 

No involvement 

16.0% 

14.8% 

Minor involvement 

40.0% 

16.4% 

Moderate involvement 

30.7% 

36.1% 

Substantial involvement 

13.3% 

32.8% 

Developing PEPs 

No involvement 

17.3% 

29.5% 

Minor involvement 

38.7% 

24.6% 

Moderate involvement 

32.0% 

24.6% 

Substantial involvement 

12.0% 

21.3% 

Providing technical support 

No involvement 

32.0% 

18.0% 

Minor involvement 

32.0% 

24.6% 

Moderate involvement 

30.7% 

27.9% 

Substantial involvement 

5.3% 

29.5% 

Other 

No involvement 

34.6% 

68.8% 

Minor involvement 

30.8% 

6.3% 

Moderate involvement 

26.9% 

12.5% 

Substantial involvement 

7.7% 

12.5% 


Source : R-Tech Principal/Facilitator Survey, spring 2009. 


130 





Table A.5 

The Challenges to R-Tech Implementation, as a Percentage of Respondents: Fall 2008 
and Spring 2009 



Fall 2008 

Spring 2009 

Challenge 

(N=153) 

(N=136) 

Communication of R-Tech goals to parents 

Not a challenge 

20.9% 

18.4% 

Minor challenge 

46.4% 

33.1% 

Moderate challenge 

25.5% 

33.8% 

Substantial challenge 

7.2% 

14.7% 

Project reporting requirements 

Not a challenge 

29.4% 

23.5% 

Minor challenge 

39.9% 

36.8% 

Moderate challenge 

23.5% 

23.5% 

Substantial challenge 

7.2% 

16.2% 

Insufficient planning time 

Not a challenge 

NA 

20.6% 

Minor challenge 

NA 

40.4% 

Moderate challenge 

NA 

22.8% 

Substantial challenge 

NA 

16.2% 

Communication of R-Tech goals to staff 

Not a challenge 

40.5% 

28.7% 

Minor challenge 

39.2% 

33.1% 

Moderate challenge 

18.3% 

27.9% 

Substantial challenge 

2.0% 

10.3% 

Development of students’ PEPs 

Not a challenge 

26.8% 

22.8% 

Minor challenge 

40.5% 

40.4% 

Moderate challenge 

28.1% 

30.1% 

Substantial challenge 

4.6% 

6.6% 

Monitoring students’ progress 

Not a challenge 

35.9% 

23.5% 

Minor challenge 

39.2% 

43.4% 

Moderate challenge 

22.9% 

25.0% 

Substantial challenge 

2.0% 

8.1% 

Coordinating training for staff 

Not a challenge 

32.0% 

23.5% 

Minor challenge 

41.8% 

44.9% 

Moderate challenge 

20.3% 

24.3% 

Substantial challenge 

5.9% 

7.4% 

Conflicts with other programs 

Not a challenge 

45.1% 

31.6% 

Minor challenge 

37.3% 

44.1% 

Moderate challenge 

13.1% 

16.9% 

Substantial challenge 

4.6% 

7.4% 


(continued) 
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Table A.5 

The Challenges to R-Tech Implementation, as a Percentage of Respondents: Fall 2008 
and Spring 2009 (continued) 


Challenge 

Fall 2008 
(N=153) 

Spring 2009 
(N=136) 

Level of technology resources 

Not a challenge 

39.2% 

32.4% 

Minor challenge 

41.2% 

40.4% 

Moderate challenge 

17.0% 

19.9% 

Substantial challenge 

2.6% 

7.4% 

Level of technical support 

Not a challenge 

43.8% 

36.8% 

Minor challenge 

40.5% 

39.7% 

Moderate challenge 

12.4% 

18.4% 

Substantial challenge 

3.3% 

5.1% 


Sources: R-Tech Principal/Facilitator Survey, fall 2008, spring 2009. 

Note. NA=Not applicable. These challenges were not included on the fall 2008 survey. 


Table A.6 

The Barriers to Student Participation in R-Tech Services, as a Percentage of Respondents: 
Fall 2008 and Spring 2009 


Barrier and Intensity 

Fall 2008 
(N=153) 

Spring 2009 
(N=136) 

Student resistance 

Not a barrier 

23.5% 

28.7% 

Minor barrier 

45.8% 

36.0% 

Moderate barrier 

26.1% 

23.5% 

Substantial barrier 

4.6% 

11.8% 

Conflicts with school-sponsored extra-curricular activities 

Not a barrier 

39.9% 

36.8% 

Minor barrier 

36.6% 

36.8% 

Moderate barrier 

21.6% 

12.5% 

Substantial barrier 

2.0% 

14.0% 

Conflicts with athletic programs 

Not a barrier 

37.9% 

40.4% 

Minor barrier 

37.9% 

28.7% 

Moderate barrier 

19.6% 

16.9% 

Substantial barrier 

4.6% 

14.0% 

Conflicts with non-school extra-curricular activities 

Not a barrier 

45.8% 

43.4% 

Minor barrier 

41.8% 

36.0% 

Moderate bander 

12.4% 

13.2% 

Substantial barrier 

0.0% 

7.4% 


(continued) 
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Table A.6 

The Barriers to Student Participation in R-Tech Services, as a Percentage of Respondents: 
Fall 2008 and Spring 2009 (continued) 


Barrier and Intensity 

Fall 2008 
(N=153) 

Spring 2009 
(N=136) 

Conflicts with student employment 

Not a barrier 

42.5% 

49.3% 

Minor barrier 

33.3% 

30.1% 

Moderate bander 

22.9% 

14.0% 

Substantial barrier 

1.3% 

6.6% 

Other 

Not a banier 

72.0% 

50.0% 

Minor banier 

4.0% 

13.6% 

Moderate banier 

4.0% 

13.6% 

Substantial barrier 

20.0% 

22.7% 

Transportation limits 

Not a banier 

42.5% 

50.0% 

Minor banier 

26.1% 

22.8% 

Moderate banier 

19.6% 

13.2% 

Substantial banier 

11.8% 

14.0% 

Parent resistance 

Not a banier 

65.4% 

58.1% 

Minor banier 

23.5% 

33.8% 

Moderate banier 

9.2% 

5.1% 

Substantial bander 

2.0% 

2.9% 


Source: R-Tech Principal/Facilitator Survey, fall 2008, spring 2009. 


Table A.7 

The Barriers to Dual Credit Implementation, as a Percentage of Respondents: Fall 
2008 and Spring 2009 


Challenge to Implementing Dual Credit 

Fall 2008 
(N=33) 

Spring 2009 
(N=29) 

Misaligned university and district semester timelines 

Not a challenge 

36.4% 

31.0% 

Minor challenge 

30.3% 

31.0% 

Moderate challenge 

27.3% 

24.1% 

Substantial challenge 

6.1% 

13.8% 

Tuition costs 

Not a challenge 

27.3% 

34.5% 

Minor challenge 

33.3% 

31.0% 

Moderate challenge 

12.1% 

24.1% 

Substantial challenge 

27.3% 

10.3% 


(continued) 
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Table A.7 

The Barriers to Dual Credit Implementation, as a Percentage of Respondents: Fall 
2008 and Spring 2009 (continued) 



Fall 2008 

Spring 2009 

Challenge to Implementing Dual Credit 

(N=33) 

(N=29) 

Textbook costs 

Not a challenge 

27.3% 

34.5% 

Minor challenge 

30.3% 

31.0% 

Moderate challenge 

27.3% 

24.1% 

Substantial challenge 

15.2% 

10.3% 

Student disinterest 

Not a challenge 

24.2% 

44.8% 

Minor challenge 

48.5% 

24.1% 

Moderate challenge 

15.2% 

27.6% 

Substantial challenge 

12.1% 

3.4% 

Coordination/ communication with university partners 

Not a challenge 

45.5% 

37.9% 

Minor challenge 

39.4% 

41.4% 

Moderate challenge 

12.1% 

20.7% 

Substantial challenge 

3.0% 

0.0% 

Student failure in dual credit courses 

Not a challenge 

NA 

41.4% 

Minor challenge 

NA 

41.4% 

Moderate challenge 

NA 

6.9% 

Substantial challenge 

NA 

10.3% 

Coordinating technical support between district and 

1 university partners 

Not a challenge 

NA 

31.0% 

Minor challenge 

NA 

51.7% 

Moderate challenge 

NA 

13.8% 

Substantial challenge 

NA 

3.4% 

Identification of appropriate course offerings 

Not a challenge 

63.6% 

34.5% 

Minor challenge 

27.3% 

55.2% 

Moderate challenge 

6.1% 

10.3% 

Substantial challenge 

3.0% 

0.0% 

Other 3 

Not a challenge 

75.0% 

50.0% 

Minor challenge 

25.0% 

16.7% 

Moderate challenge 

0.0% 

33.3% 

Substantial challenge 

0.0% 

0.0% 


Sources: R-Tech Principal/Facilitator Survey, fall 2008, spring 2009. 

Notes. The number of respondents (N) represents principals and facilitators at schools 
implementing dual credit programs. NA=Not applicable. These statements were not included on 
the fall 2008 survey. 

“Only one respondent reporting “other” challenges clarified their response within an open-ended 
item. This respondent reported the campus lacked the staff necessary to implement dual credit 
programs as they had intended. 
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Appendix B 

Online Teacher Survey 


The evaluation included a voluntary, online survey of teachers all teachers working on R-Tech campuses. 
The survey was administered in fall 2008 and again in spring 2009. The survey asked teachers about their 
roles in planning and implementing the R-Tech program, the content and format of the professional 
development teachers received in support of R-Tech, the most and least useful aspects of R-Tech training, 
their awareness of R-Tech’ s goals, and the effects of R-Tech on teachers. This appendix provides 
information about survey administration procedures, response rates, and the characteristics of survey 
respondents. The appendix also includes supplemental tables referenced in report chapters and a copy of 
the online teacher survey. 

SURVEY ADMINISTRATION 

Fall 2008 

The link to the teacher survey was sent to the principal of each R-Tech campus in November 2008. 
Principals were asked to forward the link to each teacher working on the campus. Teachers were provided 
with 3 weeks to complete the survey, and requests to remind teachers to complete the survey were sent to 
the principal. Given weak survey response rates at the survey’s deadline, evaluators kept the survey open 
for 3 additional weeks and sent additional requests to principals to encourage teacher participation. 

Spring 2009 

Identical survey procedures were used in April 2009. As in the fall, teachers were given 3 weeks to 
complete the survey, and survey reminders were sent to the principal. Again, weak response rates caused 
evaluators to keep the survey open beyond the deadline; however, due to the close of the school year and 
project timelines, evaluators were able to provide teachers only with an additional 10 days to complete the 
survey. The shortened extension period is likely the source of the reduced teacher response rate for the 
spring 2009 survey. 

Campus- and Teacher-Level Survey Response Rates 

Table B.l presents campus- and estimated teacher-level response rates for the fall and spring 
administrations of the teacher survey. In the fall 2008, 1,213 teachers working on 92 R-Tech campuses 
responded to the survey of teachers, and in spring 2009, 568 teachers on 77 R-Tech campuses responded 
to the survey. The percentage of campuses with teachers responding in fall 2008 and spring 2009 were 
80% and 67%, respectively. 

However, campus-level response rates mask the substantial variation in teacher response rates across 
R-Tech campuses. The range of teachers responding to the survey within R-Tech campuses ranged from 1 
to 56 in fall 2008, and from 1 to 42 in spring 2009. In order to gain a clearer sense of teacher response 
rates, researchers estimated teacher-level response rates using campus-level teacher counts included in 
AEIS data files for the 2007-08 school year. In terms of the percentage of teachers responding to the 
survey, teacher-level response rates are much lower — 33% in fall 2008 and 16% in spring 2009. Within 
R-Tech campuses with teachers responding to the survey, teacher response rates ranged from 1% to more 
than 100% in both the fall 2008 and spring 2009 survey administrations. Within campus response rates of 
greater than 100% arc likely the result of teachers’ aides and other support staff participating in the 


'The 2007-08 AEIS teacher counts were the most current data available at the report’s writing. 
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survey. It is unclear how valuations in teacher survey response rates across R-Tech campuses may affect 
evaluation findings, and readers are asked to use caution when interpreting survey results. 


Table B.l 

Campus and Teacher-Level Response Rates, R-Tech Teacher Survey: Fall 2008 
and Spring 2009 


Response Rate 

Fall 2008 

Spring 2009 

Campus-Level Response Rates 

Number of campuses surveyed 

115 

115 

Campuses with teachers responding 

92 

77 

Percentage of campuses with respondents 

80.0% 

67.0% 

Teacher-Level Response Rates 

Teacher counts 2007-08 

3,672 

3,620 

Number of teachers responding to survey 

1,213 

568 

Percentage of teachers responding to survey 

33.0% 

15.7% 


Sources: R-Tech Teacher Survey, fall 2008, spring 2009; Academic Excellence Indicator System 
(AEIS) data files, 2007-08 

Notes. While the number of campuses is 115 across both survey administrations, the composition 
of the 1 15 campuses changed across survey administrations. The change in the composition of 
campuses participating in 

R-Tech affected the number of teachers included in 2007-08 teacher counts across survey 
administrations. 
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CHARACTERISTICS OF SURVEY RESPONDENTS 


Table B.2 presents information about the characteristics of teachers who responded to the fall and spring 
surveys. 


Table B.2 

The Characteristics of Teacher Survey Respondents, as a Percentage of Respondents: Fall 2008 
and Spring 2009 



Fall 2008 

Spring 2009 


Middle 

High 


Middle 

High 



School 

School 

All 

School 

School 

All 


Teachers 

Teachers 

Teachers 

Teachers 

Teachers 

Teachers 

Characteristic 

(n=374) 

(n=839) 

(N=l,213) 

(n=197) 

(n=371) 

(N=568) 

Gender 

Male 

22.7% 

41.5% 

35.7% 

21.8% 

35.8% 

31.0% 

Female 

77.3% 

58.5% 

64.3% 

78.2% 

64.2% 

69.0% 

Teaching assignment: Grade 

6th grade 

41.7% 

5.4% 

16.6% 

49.2% 

5.9% 

21.0% 

7th grade 

71.4% 

11.9% 

30.3% 

69.0% 

12.9% 

32.4% 

8th grade 

66.3% 

12.4% 

29.0% 

72.1% 

14.0% 

34.2% 

9th grade 

3.2% 

74.3% 

52.3% 

3.6% 

78.7% 

52.6% 

10th grade 

1.1% 

83.4% 

58.0% 

2.0% 

86.0% 

56.9% 

1 1th grade 

0.8% 

84.5% 

58.7% 

1.5% 

87.1% 

57.4% 

12th grade 

0.5% 

81.6% 

56.6% 

0.0% 

82.7% 

54.0% 

Experience (in 
average years) 

6.4 

7.3 

7.0 

6.7 

7.0 

6.9 


Source: R-Tech Teacher Survey, fall 2008, spring 2009. 

Note. In some R-Tech districts, middle school teachers also worked in the high school, and high school teachers 
also worked in the middle school. This explains why a high school teacher may teach middle school grades and 
vice versa. 
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SUPPLEMENTAL TABLES 


The following tables present additional information cited in report chapters. 


Table B.3 

Teachers’ Roles in R-Tech Planning, as a Percentage of Respondents: Fall 2008 
and Spring 2009 


Planning Role 

Fall 2008 
(N= 1,2 13) 

Spring 2009 
(N=568) 

Decision to apply for grant 

No role 

90.7% 

87.5% 

Minor role 

6.5% 

8.3% 

Moderate role 

2.6% 

3.5% 

Substantial role 

0.2% 

0.7% 

Selection of vendors 

No role 

93.6% 

91.9% 

Minor role 

4.5% 

4.4% 

Moderate role 

1.6% 

2.8% 

Substantial role 

0.2% 

0.9% 

Other role 

No role 

96.0% 

93.0% 

Minor role 

1.9% 

3.2% 

Moderate role 

1.3% 

1.8% 

Substantial role 

0.8% 

2.0% 

Drafting the grant application 

No role 

94.9% 

93.8% 

Minor role 

4.2% 

3.5% 

Moderate role 

0.9% 

2.1% 

Substantial role 

0.0% 

0.5% 


Source : R-Tech Teacher Survey, fall 2008, spring 2009. 
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Table B.4 

Teachers’ Role in R-Tech Implementation, as a Percentage of Respondents: Fall 
2008 and Spring 2009 


Implementation Role 

Fall 2008 
(N= 1,2 13) 

Spring 2009 
(N=568) 

Supervise or monitor students 

No role 

67.0% 

54.4% 

Minor role 

11.4% 

12.1% 

Moderate role 

9.5% 

15.3% 

Substantial role 

12.1% 

18.1% 

Provide tutoring to students 

No role 

67.3% 

55.3% 

Minor role 

12.5% 

15.1% 

Moderate role 

9.6% 

13.7% 

Substantial role 

10.6% 

15.8% 

Communication with parents 

No role 

75.5% 

69.0% 

Minor role 

13.3% 

16.0% 

Moderate role 

8.2% 

8.5% 

Substantial role 

3.0% 

6.5% 

Monitor Personal Education Plans 

No role 

76.0% 

69.9% 

Minor role 

8.9% 

12.0% 

Moderate role 

8.8% 

8.8% 

Substantial role 

6.3% 

9.3% 

Identification of students 

No role 

77.9% 

71.0% 

Minor role 

11.2% 

15.7% 

Moderate role 

7.4% 

7.9% 

Substantial role 

3.5% 

5.5% 

Develop Personal Education Plans 

No role 

79.6% 

74.8% 

Minor role 

9.8% 

11.6% 

Moderate role 

6.1% 

7.7% 

Substantial role 

4.5% 

5.8% 

Identify R-Tech professional development topics 

No role 

82.5% 

77.8% 

Minor role 

12.0% 

11.8% 

Moderate role 

4.1% 

7.6% 

Substantial role 

1.4% 

2.8% 

Provide technical support 

No role 

82.7% 

78.7% 

Minor role 

12.0% 

13.4% 

Moderate role 

3.5% 

4.8% 

Substantial Role 

1.7% 

3.2% 


Source: R-Tech Teacher Survey, fall 2008, spring 2009. 
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Table B.5 

Teachers’ Views of R-Tech Goals and Outcomes, as a Percentage of Respondents, 
by Level of Agreement: Fall 2008 and Spring 2009 



Fall 2008 

Spring 2009 

Statements about Goals 

(N= 1,2 13) 

(N=568) 

Overall, 1 am pleased with the services provided by R-Tech. 

Agree/Strongly agree 

39.7% 

50.7% 

Disagree/Strongly disagree 

3.1% 

4.2% 

Don’t know 

57.3% 

45.1% 

R-Tech is positively affecting student achievement on campus. 

Agree/Strongly agree 

38.3% 

50.4% 

Disagree/Strongly disagree 

2.4% 

5.1% 

Don’t know 

59.3% 

44.5% 

Vender services are aligned with the TEKS/TAKS. 

Agree/Strongly agree 

38.4% 

47.5% 

Disagree/Strongly disagree 

2.7% 

4.4% 

Don’t know 

58.9% 

48.1% 

Goals are clearly communicated. 

Agree/Strongly agree 

35.9% 

44.5% 

Disagree/Strongly disagree 

16.7% 

18.3% 

Don’t know 

47.5% 

37.1% 

Vender services are aligned with our campus goals. 

Agree/Strongly agree 

35.4% 

42.4% 

Disagree/Strongly disagree 

3.2% 

5.8% 

Don’t know 

61.4% 

51.8% 

1 use information from student’s Personal Education Plans when 

1 plan 

classroom instruction. 



Agree/Strongly agree 

23.7% 

29.4% 

Disagree/Strongly disagree 

19.3% 

25.0% 

Don’t know 

57.0% 

45.6% 


Sources: R-Tech Teacher Survey, fall 2008, spring 2009. 

Notes. Results for teachers who agree/ strongly agree with survey items represent the sum of 
percentages of teachers who indicated they agreed or strongly agree and results for 
disagree/strongly disagree represent the sum of percentages of teachers who indicated they 
disagreed or strongly disagreed with statements on fall and spring surveys. Notably small 
percentages of teachers strongly agreed (< 12%) or strongly disagreed (< 5%) across statements 
of R-Tech goals. 
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Table B.6 

The Effects of R-Tech Implementation on Teachers, as a Percentage of Respondents, by Level of 
Agreement: Fall 2008 and Spring 2009 


Positive Effects 

Fall 2008 
(N=l,213) 

Spring 2009 
(N=568) 

1 have a greater awareness of technology-based learning opportunities for students. 

Agree/Strongly agree 

49.5% 

59.9% 

Disagree/Strongly disagree 

5.9% 

6.2% 

Don’t know 

44.6% 

34.0% 

1 have the opportunity to participate in technology-based professional development. 

Agree/Strongly agree 

47.9% 

55.6% 

Disagree/Strongly disagree 

5.1% 

8.6% 

Don’t know 

47.0% 

35.7% 

My technical skills and abilities have improved. 

Agree/Strongly agree 

43.4% 

53.2% 

Disagree/Strongly disagree 

7.4% 

9.2% 

Don’t know 

49.1% 

37.3% 

1 have a better understanding of the needs of at-risk students. 

Agree/Strongly agree 

41.5% 

50.5% 

Disagree/Strongly disagree 

10.3% 

11.8% 

Don’t know 

48.2% 

37.7% 

My teaching has improved. 

Agree/Strongly agree 

38.3% 

47.2% 

Disagree/Strongly disagree 

7.6% 

9.7% 

Don’t know 

54.1% 

43.1% 

My lesson plans have improved. 

Agree/Strongly agree 

36.2% 

43.5% 

Disagree/Strongly disagree 

11.4% 

13.7% 

Don’t know 

52.4% 

42.8% 


Source: R-Tech Teacher Survey, fall 2008, spring 2009. 

Notes. Results for teachers who agree/ strongly agree with survey items represent the sum of percentages of 
teachers who indicated they agreed or strongly agreed and results for disagree/strongly disagree represent the sum 
of percentages of teachers who indicated they disagreed or strongly disagreed with statements on fall and spring 
surveys. Notably small percentages of teachers strongly agreed (< 14%) or strongly disagreed (< 3%) across 
statements about R-Tech effects. 
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Appendix C 

Online Student Survey 


In order to gain students’ views about R-Tech services, the evaluation included a voluntary, online survey 
of students who participated in R-Tech during the 2008 summer session, fall 2008, and/or spring 2009 
semesters. The survey asked students about their access to R-Tech services, their perceptions of R-Tech 
technology resources, the subject areas they studied using R-Tech, as well as what they liked most and 
least about using technology resources for learning. Students participating in dual credit coursework were 
routed to a separate survey section addressing the courses taken for dual credit and students’ perceptions 
of course rigor and content. This appendix contains information about the administration of the student 
survey, survey response rates, the characteristics of survey respondents, and a copy of the online student 
survey administered in spring 2009. 

SURVEY PROCEDURES 

Parental Consent 

Three weeks in advance of survey administration, principals on 1 15 R-Tech campuses were asked to 
distribute a letter to parents of students participating in R-Tech services during the 2008 summer session, 
fall 2008, and/or spring 2009 semesters requesting parent consent for students to participate in the survey. 
The letter explained survey content and provided parents with a link that enabled them to preview the 
survey. The letter clarified that participation in the survey was voluntary and that students who did not 
participate in the survey would not experience any penalties. Parents who did not want their students to 
participate in the survey were asked to return a signed form to TCER or to notify TCER that they did not 
approve of their students’ participation by telephone or e-mail. TCER does not have information about 
how many parents received consent forms; however, five parents returned forms indicating that they did 
not want their students to participate in the spring survey. Principals received notification of the students 
whose parents requested that they not participate in the survey in advance of survey administration. 

Survey Administration 

In spring 2009, principals were provided with a link to an online survey of students receiving R-Tech 
services during the 2008 summer session, the fall 2008, and/or spring 2009 semesters. Principals were 
asked to forward the link to a convenient location for students to access the survey (e.g., a computer lab). 
Principals on campuses attended by students whose parents had returned forms indicating that they did 
not want their child to participate in the survey also received a list of students who should not participate 
in the survey. Students were given 3 weeks to complete the survey. However, weak response rates at the 
survey’s deadline caused evaluators to keep the survey open for 10 additional days, and principals were 
sent reminders to encourage student participation. 
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Number of Survey Respondents and Response Rates, Spring 2009 


Table C.l presents campus- and student-level response rates for spring 2009 online student survey, and 
indicates that 2,993 students attending 54 campuses responded to the survey. 1 Less than half of R-Tech 
campuses (47%) had students who participated in the survey. Using student data upload information 
submitted by districts to TEA for the summer 2008, fall 2008, and spring 2009 reporting periods, TCER 
identified 14,849 unique students who received R-Tech services during the program’s first year, and 
calculated a student-level survey response rate (i.e., the percentage of students receiving R-Tech services 
who participated in the survey) of approximately 20%. Across campuses with students participating in the 
survey, the number of survey respondents ranged from 1 to 363 students. It is unclear how variations in 
the number of students responding to the survey across R-Tech campuses may affect evaluation findings, 
and readers are asked to use caution when interpreting survey results. 


Table C.l 

Campus- and Student-Level Response Rates, R-Tech Student Survey: 
Spring 2009 


Response Rate 

Spring 2009 

Campus-Level Response Rates 

Number of campuses surveyed 

115 

Campuses with students responding 

54 

Percentage of campuses with respondents 

47.0% 

Student-Level Response Rates 

Number of students participating in R-Tech a 

14,849 

Number of students responding to survey 

2,993 

Percentage of students responding to survey 

20.1% 


Sources'. R-Tech Student Survey, spring 2009; Texas Education Agency, Student 
Upload data: summer 2008, fall 2008, and spring 2009 

a Total number of unique students who received R-Tech services in at least one of 
the following periods: summer 2008, fall 2008, and spring 2009. Students receiving 
services across multiple periods are counted only once. 


'in total, 3,047 students responded to the online survey; however, 54students indicated that they did not agree to 
complete the survey. These students were exited from the survey and no further data were collected. The student- 
level response rate presented in Table C.l is based on the 2,993 students who provided responses to the survey. 
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CHARACTERISITICS OF SURVEY RESPONDENTS 


Table C.2 provides information about the characteristics of students who participated in the spring 2009 
survey. Results are presented for middle school students, high school students, and all respondents. 


Table C.2 

The Characteristics of Student Survey Respondents, as a Percentage of 
Respondents: Spring 2009 



Spring 2009 


Middle School 

High School 

All 


Students 

Students 

Respondents 

Characteristic 

(n=l,530) 

(n= 1,463) 

(N=2,993) 

Gender 

Male 

51.8% 

52.2% 

52.0% 

Female 

48.2% 

47.8% 

48.0% 

Ethnicity 

African American 

8.0% 

9.3% 

8.7% 

Hispanic/Latino 

47.5% 

32.2% 

40.0% 

White 

39.7% 

54.8% 

47.1% 

Other 1 * 

4.8% 

3.8% 

4.3% 

Grade Level 

6th grade 

24.3% 

— 

12.4% 

7th grade 

32.5% 

— 

16.6% 

8th grade 

43.1% 

— 

22.1% 

9th grade 

— 

23.9% 

11.7% 

10th grade 

— 

27.8% 

13.6% 

1 1th grade 

— 

28.4% 

13.9% 

12 th grade 

— 

19.9% 

9.7% 


Source: R-Tech Student Survey, spring 2009. 

a Students entering written responses for “Other” identified themselves as 
“multiracial,” “Asian,” “Chinese,” “Korean,” “Indian,” “American,” “Scandinavian,” 
“Pacific Islander,” “German,” “ Native American,” “Jewish,” “brown,” and “unsure.” 
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Appendix D 

R-Tech Site Visits 


In order to gain a more holistic understanding of how R-Tech is implemented in Cycle 1 grantee districts, 
as well as how implementation designs may affect teacher and student outcomes, the evaluation included 
site visits to eight R-Tech districts (13 campuses) conducted in April 2009. Site visit districts were 
selected such that each category of R-Tech program identified by the evaluation was represented (i.e., 
self-paced programs, dual credit programs, technology immersion programs, and programs providing 
iPods loaded with instructional content), 1 and that programs varied in terms of size, location, types of 
students served (e.g., percentages of low income or ELLs), and R-Tech vendor selections. Researchers 
also confirmed that site visit districts had fairly well established programs that had been serving students 
across the fall 2008 and the spring 2009 grant periods. In addition, researchers identified alternate districts 
offering similar programs in case any of the initially identified districts declined to participate in site 
visits. 

In January 2009, TCER sent an e-mail to district- and campus-level administrators in identified districts, 
inviting participation in the R-Tech site visits. The e-mail described site visit activities and clarified that 
visits were designed to enhance evaluation findings and that site visit data would not be used to monitor 
districts’ compliance with grant requirements. TCER researchers followed up with phone calls to answer 
questions and encourage participation in the visits. TEA provided support, assuring districts that site visits 
were a valuable component of the evaluation and that researchers were not compliance monitors. As a 
result, all of the initially identified eight districts agreed to participate in the site visits. The following 
sections provide information about site visit activities and the types of R-Tech programs offered by site 
visit districts. 

SITE VISIT ACTIVITIES 

In site visit districts, researchers visited each campus implementing R-Tech (e.g., middle school and high 
school) and conducted interviews with campus principals and R-Tech facilitators and held focus group 
discussions with teachers involved in the grant. Researchers also conducted focus group discussions with 
students who received R-Tech services on campuses in which more than three students had returned 
materials indicating parental consent to participate in discussions. The process for obtaining informed 
consent is discussed in the section describing student focus groups discussions. In districts offering dual 
credit programs, a separate focus group was held for students enrolled in dual credit courses. In addition, 
researchers observed delivery of R-Tech services; however, in one district, services were provided in 
students’ homes, which prevented researcher observation. 

In collaboration with TEA, TCER developed interview and focus group discussion protocols, as well as 
an observation instrument designed to collect information about R-Tech service delivery. Protocols were 
reviewed and approved by TEA’s Data Information and Review Committee (DIRC) in March 2009. Site 
visits schedules were developed in coordination with district staff, and all visits were completed in April 
2009. The sections that follow provide information on each site visit activity. 

The Principal Interview 

The principal interview gathered information about the grant application processes, program planning, the 
roles and responsibilities of staff involved in implementing R-Tech, as well as the challenges of 
implementing R-Tech and how challenges may have been overcome. The interview also addressed 


'Neither district offering one-to-one tutoring with online instructional support served students during R-Tech’ s first 
implementation year. 
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R-Tech’s effects on student and teacher outcomes and program sustainability. Principal interviews were 
conducted on each site visit campus. In one district, the principals of both campuses participating in the 
grant were unavailable on the day of the site visit, and interviews were conducted by telephone at a later 
date. 

The R-Tech Facilitator Interview 

Similar to the principal interview, the program facilitator interview asked about grant planning and 
implementation processes, the roles of staff in implementing R-Tech and the challenges and benefits of 
participation in R-Tech. The facilitator interview also asked about the training facilitators may have 
received in support of the grant. Researchers conducted facilitator interviews in all R-Tech districts. 

Teacher Focus Group Discussions 

Using data provided by districts through TEA’s R-Tech Teacher Upload for the fall 2008 grant period, 
researchers identified teachers involved in the program to participate in focus group discussions. 
Researchers requested the schools’ master schedules, which provide information about teacher schedules 
and conference periods, in advance of site visits. To the extent possible, researchers scheduled focus 
group discussions during a common conference period or planning time. In several districts, however, 
teachers did not have common free periods and focus groups were held after school. Researchers apprised 
teachers that participation in the focus group was voluntary. During discussions, researchers probed 
teachers’ roles in planning and implementing R-Tech; their involvement in grant activities, including 
professional development; and the challenges and benefits teachers and students experience as a result of 
the grant. 

Student Focus Group Discussions 

Prior to scheduled site visits, TCER sent principals at each site visit campus a packet containing a letter to 
parents requesting permission for students to participate in focus group discussions, parental consent 
forms, and a postage paid envelope in which parents could return signed consent forms to TCER. TCER 
requested that principals distribute materials to students who participated in R-Tech services during the 
fall 2008 or spring 2009 semesters, and that students deliver materials to parents. The parent letter 
clarified that participation in the discussion was voluntary and students would not be penalized for 
choosing not to participate. It indicated that focus group discussions would be recorded, but that students 
would not be identified by name and responses would remain confidential. The letter provided 
information about the types of questions researchers would ask during focus groups and requested “active 
parent consent” for student participation in discussions. Parents indicated consent by returning a signed 
consent form to TCER in the provided postage paid envelope. The number of parents who returned signed 
consent forms varied from 1 to 15 across site visit campuses. 

Researchers scheduled student focus groups at campuses in which the parents of three or more students 
returned signed consent forms. At campuses in which consent was provided for more than eight students, 
researchers randomly selected eight students to participate in focus groups. Separate focus groups were 
scheduled for students participating in dual credit coursework. Researchers provided campus principals 
with a list of students identified for focus groups and asked principals to facilitate the activity by 
providing a space for the focus group (e.g., a conference room) and releasing students from class at the 
scheduled time for the discussion. At the start of focus groups, researchers advised students that their 
parents had provided consent for their participation in the discussion, but that students’ participation was 
voluntary. Researchers asked students to describe what they did using R-Tech resources and the locations 
and times that they accessed resources. Researchers asked whether learning with R-Tech resources was 
different than traditional classroom instruction and what students liked most and least about using 
technology resources for learning. Students participating in dual credit courses responded to questions 
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about the format of dual credit instruction, the courses they took, how dual credit courses differed from 
regular high school coursework, and the challenges and benefits students experienced as a result of their 
participation in dual credit courses. 

Observation of R-Tech Services 

With the exception of District C, in which students participated in R-Tech services at home (see program 
descriptions in the next appendix section), researchers observed the delivery of R-Tech services at each 
site visit campus. Observations documented the number of students receiving services, when and where 
services were delivered, teacher or facilitator roles and the types of materials involved in providing 
services, as well as students’ engagement in learning. 

SITE VISIT DISTRICTS 

Table D.l provides an overview of the types of programs provided by districts that participated in site 
visits, the grade levels served, as well as information about site visit activities. The sections that follow 
provide more information about the R-Tech programs offered by individual districts. 


Table D.l 

Overview of R-Tech Site Visit District Programs 


District 

Grade 

Levels 

Served 

Program Type 

Site Visit Notes 

A 

7-12 

Self-paced software providing credit recovery 
and TAKS remediation; dual credit coursework 
(Grades 11 and 12) 

All site visit activities completed at 
combined MS/HS campus. No 
focus group with dual credit 
students. 

B 

6-12 

Self-paced software providing credit recovery 
and TAKS remediation. 

All site visit activities completed at 
MS and HS. 

C 

6-8 

Instructional content loaded on iPods 

Unable to observe R-Tech service 
delivery (provided at students’ 
homes); all other activities 
completed. 

D 

6-9 

Self-paced TAKS remediation 

Principal interviews conducted by 
telephone; all other activities 
completed during site visit. 

E 

7-8 

Self-paced computer software; TAKS 
remediation and credit recovery 

All site visit activities completed at 
participating campus. 

F 

6-12 

School-wide technology immersion program; 
Dual credit coursework (Grades 1 1 and 12) 

All site visit activities completed at 
MS and HS, including focus group 
for students participating in dual 
credit coursework. 

G 

6-12 

Self-paced remediation in ELA 

All site visit activities completed at 
MS and HS. 

H 

6-12 

Self-paced program focused on social and 
behavioral modification 

No student focus groups (MS or 
HS); all other activities completed. 


Sources: District grant applications and progress reports; site visit data. 
Notes. HS=High School; MS=Middle School. 
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District A 


District A is one of the smallest districts participating in R-Tech, and is located in a remote agricultural 
community with few resources to support student learning beyond those provided by the district. A 
substantial proportion of District A’s students come from economically disadvantaged backgrounds, and 
many students are characterized as “at risk” of academic failure. District A implements R-Tech as a 
self-paced program, providing credit recovery, credit acceleration, and online tutorials for students in 
Grades 7 through 12 using Web-based software. In addition, the district offers a dual credit program in 
coordination with the regional ESC for students in Grades 1 1 and 12. Middle school and high school 
students attend the same campus, and the district used R-Tech funding to update and expand access to the 
campus’ computer lab. The district initially planned to provide access to R-Tech services after school and 
on Saturdays; however, students’ unwillingness to participate in services outside of school caused the 
district to revise its plans to include R-Tech services as part of the school day. In addition, the district uses 
R-Tech resources to provide tutoring in preparation for TAKS testing during regularly scheduled class 
time. Dual credit coursework is provided through the high school’s distance learning lab using video 
conferencing equipment. 

District B 

District B is located in a small community that lies outside a large metropolitan area. Most district 
students are White, and few come from economically disadvantaged backgrounds. However, school 
administrators became increasingly concerned by poor TAKS performance and high dropout rates among 
the district’s low income and minority students, and noted that few such students had access to 
technology resources outside the classroom. While the district had a computer lab in place prior to 
R-Tech, it lacked the resources to staff the lab, and the lab remained closed during the school day and 
after school. 

The district used R-Tech funds to open a second “R-Tech” lab furnished with new hardware and to hire a 
full-time lab facilitator. The lab is accessible to both middle and high school students and R-Tech is 
implemented as a TAKS remediation program in the core content areas for students in Grades 6 through 
12 using a self-paced software program purchased prior to the district’s R-Tech application. Although the 
source of confusion is unclear, school administrators understood that R-Tech required students to have 
access to services 10 hours a day rather than 10 hours a week. As a means to ensure extended access to its 
lab each day, the full-time facilitator provides supervision during the school day, and teachers staff the 
R-Tech lab before and after school. In spite of expanded access to the lab. District B implements its 
R-Tech program as paid of its regular school day. Students who have performed poorly on TAKS arc 
assigned to receive R-Tech services as part of a TAKS preparation class that takes the place of an elective 
course. In addition, core content area teachers use R-Tech resources to support TAKS preparation by 
scheduling class time in the lab. However, teachers note that all students benefit from expanded access to 
the lab because they are able use computers to complete homework assignments that require the use of 
online resources. 

District C 

District C is located in a rural, farming community experiencing an influx of Hispanic students, many of 
whom are characterized as ELL or LEP. As a means to support students’ development of English 
language skills, the district used R-Tech resources to purchase iPods, which it provided to ELL students 
in Grades 6 through 8. Teachers loaded iPods with instructional resources targeted to students learning 
English, including podcasts addressing grammar and vocabulary, and lessons across the four core content 
areas. Students take iPods home and access podcasts outside of the regular school day. Many students use 
iPod resources with family members, which administrators note has improved the language skills of 
parents and siblings. Teachers received vendor-provided training in how to find and create instructional 
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videos and integrate their use in the classroom. In the future, district representatives hope to expand the 
program into all classrooms at the middle and high school levels. The district’s primary R-Tech objective 
is to improve TAKS scores among its ELL and LEP students. 

District D 

District D is in a small community located at the fringe of large metropolitan area. The district enrolls a 
large proportion of low income students, and nearly all students are White. Many of the district’s students 
reside in a local children’s home and face academic challenges resulting from homelessness, poverty, and 
parental neglect. District administrators note that high poverty levels and low property values make 
District D one of the poorest school districts in the state. 

District D is seeking to improve its TAKS scores through R-Tech and used grant funding to implement an 
online program that provides self-paced remediation in core content areas and tutorials focused on TAKS 
preparation in its middle school and high school (Grades 6 through 9). The district purchased additional 
computers for its computer labs and enabled students to access the program before and after school, and 
during free periods. Students with home Internet access may connect to R-Tech resources, and the district 
supported the children’s home in expanding its Internet access to enable students to use the program. 
Students assigned to the district’s alternative placement program also use R-Tech resources. 

District E 

District E is one of the largest districts participating in R-Tech and is located in a “bedroom community” 
at the outskirts of a large urban area. The district enrolls large proportions of low income and minority 
students, many of whom struggle with ELA and math content and have failed portions of the TAKS test. 
District E used R-Tech funding to implement a self-paced tutoring and remediation program focused on 
math and ELA for a targeted group of low performing students in Grades 7 and 8. The school implements 
an 8-period school day, which provides students with an extra period for an elective or study hall. 

Students identified for R-Tech are scheduled to receive services in the computer lab during their extra 
period, and students may also access services after school. 

District F 

District F is located in the county seat of a rural community focused on manufacturing and oil and gas 
production. The district is experiencing changes in the makeup of its student population, and enrolls 
increasing numbers of minority and low income students. District administrators were concerned with the 
“significant” achievement gap they observed for the growing number of minority and low income 
students, and sought R-Tech funding to implement a technology immersion program that would address 
students’ individual learning styles and provide the technical skills needed by the region’s employers. 

District F used R-Tech funds to purchase laptops for all students and teachers in Grades 6 through 12. 
Students are issued their own laptops which they use in class and at home. Teachers receive continuing 
training in the integration of technology in classroom instruction and use laptops to prepare and deliver 
course content and to tailor instruction to individual student needs. The district also implements a dual 
credit program in its high school. The dual credit program is offered using two-way video conferencing 
equipment located in the high school’s distance learning lab. Students participating in dual credit 
coursework use laptops to take notes and to communicate online with course instructors 

District G 

District G is located in a rural, manufacturing community in the vicinity of a mid-sized urban area. The 
district enrolls a predominantly minority student population, many of whom are from low income 
backgrounds. District administrators recognized a longstanding pattern of poor achievement in reading 
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comprehension among students, and after administering a survey, realized that most students did not have 
access to home computers that could supplement learning. The district used R-Tech funding to purchase a 
self-paced remediation program targeted to ELA. All students in Grades 6 through 12 use the program as 
part of regular instruction in English. Students read non-fiction texts and current event articles and 
respond to a series of questions about what they have read. The program is intended to improve students’ 
vocabulary and comprehension skills. Students may access the program outside of school hours for 
additional support by checking out laptops or using workstations at the public library. 

District H 

District H is located in a rural community with roots in ranching, tourism, and outdoor recreation. Most of 
the district’s students are White and from low income backgrounds, and many are at risk of academic 
failure. A school administrator explained that many district students come from families marked by 
“chronic generational underachievement” and that “parents are not equipped to model appropriate 
behaviors and reinforce their use.” Recognizing that “psycho-social factors affect achievement,” district 
administrators implemented R-Tech as a social and behavioral program designed to increase attendance 
rates and academic achievement for its middle and high school students 

The district used R-Tech resources to purchase a self-paced software program that promotes empathy, 
problem solving, connectedness, assertiveness, and pro-social behavior. The program is available to all 
students in Grades 6 through 12. At-risk middle school students are assigned to use the program either 
before or after school, and high school students access the program when they are assigned to after school 
detention or Saturday school because of disciplinary referrals. High school students are required to 
complete academic work first (e.g., homework) and then may use R-Tech resources. 


152 



Appendix E 

The Implementation Of Dual Credit Courses On R-Tech Campuses 


The legislation enabling R-Tech allows participating districts to use grant funding to implement “distance 
learning opportunities that enable students to earn college credit in the subject areas of English language 
arts, social studies, mathematics, science or languages other than English” (TEC § 29.919 [d][5]). Dual 
credit course arrangements enable 11th- and 12th-grade students' to simultaneously earn high school and 
college credit by “successfully completing a college course that covers all the TEKS of any specified high 
school course” (TEA, 2007). Students participating in dual credit courses may not be required to pay 
tuition costs or to purchase texts used in college classes. 

Some research has indicated that technology-based dual credit opportunities provide rural districts with a 
cost-effective means of diversifying course offerings and increasing academic rigor (Malhoit, 2005). 
Participation in dual credit courses may also provide encouragement for students to pursue postsecondary 
educational opportunities, and offset education costs by enabling students to earn college credit while 
receiving a publicly funded high school education (Maloney, Lain, & Clark, 2009). As indicated in 
chapter 3, 18 of the 62 districts (29%) that implemented R-Tech during the program’s first year offered 
dual credit coursework. This appendix provides information about the dual credit programs implemented 
as part of R-Tech. It describes the characteristics of students participating in dual credit courses, the 
challenges districts experience in implementing dual credit courses, and students’ perceptions of course 
offerings. 

DATA SOURCES 

This appendix draws on data provided to TEA through the Agency’s R-Tech data upload system as well 
as from PEIMS data. In addition, analyses present findings from the fall and spring surveys of principals 
and program facilitators in R-Tech districts, as well as from the spring survey of students participating in 
R-Tech services. More information on the survey administration process, response rates, respondent 
characteristics, and supplemental tables providing additional information on findings cited in this 
appendix may be found in Appendix A (principal and facilitator survey) and Appendix C (student 
survey). 

STUDENTS’ PARTICIPATION IN DUAL CREDIT PROGRAMS 

The following sections present information about the students participating in R-Tech dual credit 
programs, including the demographic characteristics and the subject areas students addressed through 
dual credit coursework. Students identified as participants in dual credit courses were included in 
districts’ student upload data for the fall 2008 and spring 2009 reporting periods 2 and upload reports 
indicated that their primary instructional area was “Distance Learning to Earn College Credit.” Of the 
3,390 students in Grades 1 1 and 12 included in the data upload for the fall and spring reporting periods, 
232 (6.8%) were enrolled in R-Tech dual credit courses during the 2008-09 school year. 


'Students enrolled in Texas’ early college high schools may participate in some dual credit courses as early as the 
ninth grade. 

2 No students participated in R-Tech funded dual credit coursework as part of the 2008 summer session. 
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The Characteristics of Dual Credit Students 


Table E. 1 presents the characteristics of 1 1th- and 12th-grade students who participated in R-Tech dual 
credit programs during the 2008-09 school year. Results indicate that relative to other 1 1th- and 12th- 
grade R-Tech students, dual credit students were less likely to be African American (3% vs. 9%) or from 
low income backgrounds (27% vs. 43%), and were more likely to be female (58% vs. 47%) and White 
(70% vs. 62%). In addition, proportionately fewer special education (0% vs. 13%) and LEP (0% vs. 3%) 
students participated in dual credit courses relative to other R-Tech participants. 


Table E.l 

The Characteristics of Students Participating in R-Tech Dual 
Credit Programs, Grades 11 and 12: Fall 2009 and Spring 2008 


Student Group 

Dual Credit a b 
(N=232) 

Non-Dual 

Credit cd 

(N=3,158) 

African American 

2.7% 

9.4% 

Hispanic 

26.5% 

27.8% 

White 

69.9% 

61.7% 

Other 

0.8% 

11% 

Female 

58.0% 

47.1% 

Male 

53.4% 

51.7% 

Economically disadvantaged 

27.0% 

43.2% 

Special education 

0.4% 

13.3% 

Limited English proficient 

0.0% 

2.7% 


Sources: Public Education Information Management System fall 2007 
snapshot data for the students attending the 115 participating campuses; Texas 
Education Agency R-Tech Student Upload data, fall 2008, and spring 2009. 
“These students had reported fall 2008 or spring 2009 primary instructional 
hours greater than 0 and their primary instructional method in fall 2008 or 
spring 2009 was distance learning to earn college credit. 
b There were 6 of the 232 students who had missing demographic information. 
The percentages in the table were based on the 226 students who had 
demographic information. 

“These students had reported fall 2008 or spring 2009 primary instructional 
hours greater than 0 and their primary instructional method in fall 2008 or 
spring 2009 was research-based instructional support, academic tutoring or 
counseling, distance learning aligned with the TEKS, Web-based program, or 
Apangea lecture. 

d There were 143 of the 3,158 students who had missing demographic 
information. The percentages in the table were based on the 3,015 participants 
who had demographic information. 
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The Subject Areas Studied by Dual Credit Students 

Table E.2 presents information about the subject areas 1 1th- and 12th-grade students participating in 
R-Tech dual credit programs studied relative to other 1 1th- and 12th-grade students receiving R-Tech 
services. Results indicate some notable differences in the subject areas emphasized in dual credit 
programs. Relative to 1 1th- and 12th-graders participating in other R-Tech programs, students in dual 
credit programs were considerably more likely to focus on social studies (60% vs. 10%) and ELA (40% 
vs. 27%). In contrast, students in other programs were more likely to focus on math (50% vs. 7% for dual 
credit students) and science (19% vs. 3%). The reasons for these differences were not clear in spring 
2009. 

Table E.2 

The Subject Areas Addressed by Students Participating in R-Tech 
Dual Credit Programs, Grades 11 and 12: Fall 2009 and Spring 
2008 


Subject Area 

Dual Credit a b 
(N=232) 

Non-Dual 

Credit c,b 

(N=3,158) 

Mathematics 

7.3% 

49.5% 

English language arts 

39.7% 

26.7% 

Science 

3.4% 

18.5% 

Social studies 

59.9% 

9.9% 

Language other than English 

0.0% 

0.8% 

Other subject area 

0.0% 

0.1% 


Sources: Public Education Information Management System fall 2007 snapshot 
data for the students attending the 115 participating campuses; Texas Education 
Agency R-Tech Student Upload data, fall 2008, and spring 2009. 

“These students had reported fall 2008 or spring 2009 primary instructional 
hours greater than 0 and their primary instructional method in fall 2008 or 
spring 2009 was distance learning to earn college credit. 
b Percentages total to more than 100% because a student could have a different 
primary academic area in fall 2008 and spring 2009. 

“These students had reported fall 2008 or spring 2009 primary instructional 
hours greater than 0 and their primary instructional method in fall 2008 or 
spring 2009 was research-based instructional support, academic tutoring or 
counseling, distance learning aligned with the TEKS, Web-based program, or 
Apangea lecture. 
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THE IMPLEMENTATION OF TECHNOLOGY-BASED DUAL CREDIT COURSES 


The fall and spring survey of principals and R-Tech facilitators contained items addressing the provision 
of technology-based dual credit courses. Survey respondents were asked if their schools offered dual 
credit instruction as part of R-Tech. Principals and facilitators who responded “Yes” were routed to a 
separate set of questions addressing the challenges of implementing dual credit courses as well as how 
challenges were overcome. In fall 2008, 33 of 153 survey respondents (22%) indicated that their schools 
offered a dual credit program. In spring 2009, 29 of 136 survey respondents (21%) reported offering dual 
credit courses as paid of R-Tech. 

The Challenges to Implementing Technology-Based Dual Credit Programs 

The survey provided principals and facilitators with a list of common challenges to implementing dual 
credit courses and asked respondents to indicate whether challenges were a minor , moderate or 
substantial barrier to implementation or whether challenges were not a barrier. Table E.3 presents the 
summed percentages of survey respondents who considered each challenge to be a moderate or 
substantial barrier. Summed percentages are the percentage of respondents who indicated moderate 
challenges plus the percentage of respondents who indicated substantial challenges. (See Table A.7 in 
Appendix A for percentages by each response category for each survey item across both survey 
administrations). Response patterns indicate that as dual credit programs got started in the fall, principals 
and administrators considered tuition and textbook costs to be the greatest barrier; however, as the year 
progressed, respondents were more concerned with the lack of alignment between college and district 
calendars. Differing calendars may create challenges for districts when colleges have different semester 
start/end dates, holidays, exam dates, and grading periods. Although the percentages of survey 
respondents who felt textbook and tuition costs were challenges to dual credit courses declined across 
survey administrations, more than a third of respondents still indicated these expenses were moderate or 
substantial challenges in spring 2009. Although the state provides funding for high school textbooks, it is 
not obligated to pay for materials for students’ college courses, and districts must allocate funds for 
students’ college texts. 


Table E.3 

Principals’ and Facilitators’ Perceptions: Moderate and Substantial Barriers to Dual Credit 
Implementation, as a Summed Percentage of Respondents: Fall 2008 and Spring 2009 


Moderate/Substantial Challenges to Implementing Dual Credit Courses 

Fall 2008 
(N=33) 

Spring 2009 
(N=29) 

Misaligned university and district semester timelines 

33.4% 

37.9% 

Tuition costs 

39.4% 

34.4% 

Textbook costs 

42.5% 

34.4% 

Student disinterest 

27.3% 

31.0% 

Coordination/ communication with university partners 

15.1% 

20.7% 

Student failure in dual credit courses 

NA 

17.2% 

Coordinating technical support between district and university partners 

NA 

17.2% 

Identification of appropriate course offerings 

9.1% 

10.3% 

Other 

0.0% 

33.3% 


Sources: R-Tech Principal/Facilitator Survey, fall 2008, spring 2009. 

Notes. Summed percentages represent total of two response categories: (1) the percentage of respondents who 
indicated moderate challenges, and (2) the percentage of respondents who indicated substantial challenges. 

The number of respondents (N) represents principals and facilitators at schools implementing dual credit programs. 
Results will not total to 100; respondents could indicate multiple challenges. NA=Not applicable. These statements 
were not included on the fall 2008 survey. 
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The survey also provided respondents with space to enter open-ended comments describing other 
challenges. While a third of spring respondents indicated other challenges, only one person entered a 
written response. The response indicated that the district lacked the necessary staff to implement dual 
credit courses effectively. 

Overcoming Challenges to Dual credit Implementation 

The spring survey also provided respondents with an opportunity to indicate how challenges to 
implementing dual credit courses may have been overcome. The survey presented respondents with a list 
of strategies and asked them to mark all that may have been used to support the implementation of dual 
credit instruction. As presented in Table E.4, a majority of survey respondents (66%) indicated that they 
worked with university staff to resolve problems, such as calendar conflicts and communication issues. 
Many respondents reported using HSA and local funds to pay textbook and tuition costs. A large 
proportion of principals and facilitators (4 1 %) supported students enrolled in dual credit courses by 
providing additional tutoring and academic counseling. 3 An equal proportion of respondents (41%) held 
informational sessions to increase student interest and awareness of dual credit opportunities. 


Table E.4 

Districts’ Methods to Overcoming Challenges to Dual Credit Implementation, 
as a Percentage of Survey Respondents: Spring 2009 


Method 

All Respondents 
(N=29) 

Collaborated with university staff to resolve challenges 

65.5% 

Used HSA or other funds to support costs 

44.8% 

Provided additional academic support for students 

41.4% 

Held information sessions for students and parents 

41.4% 

Adjusted district calendar to accommodate university timelines 

13.8% 

Other 

17.2% 


Source : R-Tech Principal/Facilitator Survey, spring 2009. 

Note. The number of respondents (N) represents the principals and facilitators implementing 
dual credit courses. Percentages will not total to 100 because respondents could select more 
than one method. 

STUDENTS’ PERCEPTIONS OF DUAL CREDIT COURSES 

The spring 2009 student survey asked respondents if they had taken a technology-based course that 
enabled them to receive college credit during R-Tech’ s first year. The survey routed students who had 
participated in R-Tech dual credit offerings to a separate set of questions asking about the courses they 
took and their perceptions of dual credit offerings. Of the 707 students in the eleventh and twelfth grades 
who responded to the spring survey, 171 (24%) responded that they participated in dual credit courses as 
part of R-Tech. 

Student-Reported Dual Credit Subject Area 

The survey provided students with space to enter the courses they took as part of dual credit programs, 
and researchers categorized answers by common subject area. Table E.5 presents students’ responses, and 
similar to findings presented in Table E.2, results indicate that most surveyed students (54%) focused on 
subject areas related to social studies (e.g., history, government, and economics). Many students also took 
courses related to English (49%), and notably small percentages of students took dual credit courses 


3 High school students who fail dual credit courses may jeopardize their eligibility for financial aid or to participate 
in college sports programs because dual credit courses are college classes. 
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related to math (9%) or science (6%). Surveyed students also reported participating in dual credit courses 
related to psychology and sociology (15%) or business related courses (15%). Students entering “Other” 
courses reported taking courses such as art appreciation, journalism, criminal justice, and courses focused 
on the use of technology. 


Table E.5 

Subject Areas Addressed in Dual Credit Courses, as a Percentage of 
Survey Respondents: Spring 2009 


Subject Area 

All Respondents 
(N=171) 

English 

49.1% 

History 

28.7% 

Government/Economics 

24.0% 

Psychology/Sociology 

15.2% 

Business and business management 

14.6% 

Math 

9.4% 

Science 

5.8% 

Languages 

4.1% 

Other 

9.4% 


Source'. R-Tech Student Survey, spring 2009. 

Note. Percentages will not total 100; students could list more than one course name. 


Students’ Perceptions of Dual Credit Coursework 

The survey provided students with a list of statements about dual credit courses and asked them to rate 
their level of agreement with each statement. Researchers coded their responses: strongly disagreed (-10), 
disagreed (-5), unsure (0), agreed (5), and strongly agreed (10) as a means to clearly illustrate variations 
in levels of agreement. Table E.6 presents students’ mean responses. Values closer to 10 indicate higher 
levels of agreement and values closer to -10 indicate higher levels of disagreement. 


Table E.6 

Students’ Perceptions of Participation in Dual Credit Courses, as a Mean of Respondents: Spring 
2009 


Statement 

All Respondents 
(N=171) a 

Dual credit coursework is preparing me for college. 

5.0 

I would like to take more dual credit courses using technology resources. 

3.7 

The calendar for my course is different from my regular classes. 

3.4 

Dual credit courses are more challenging than regular courses. 

3.2 

It is easy to communicate with other students. 

2.9 

It is easy to communicate with the instructors. 

2.5 

I can get help easily when I don’t understand information. 

1.8 

I had difficulty obtaining the textbook and other materials. 

-1.6 


Source: R-Tech Student Survey, spring 2009. 

Notes. Items have been coded: strongly disagree (-10), disagree (-5), unsure (0), agree (5), strongly agree (10). 
Items closer to 10 indicate higher levels of agreement with each statement. 

“The number of respondents represents students enrolled in dual credit courses during the 2008-09 school year. 

Responses suggest that R-Tech dual credit offerings positively enhance students’ high school experience 
and provide access to rigorous course content. Students expressed the highest level of agreement with the 
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statement indicating dual credit courses provided preparation for college. Responses also indicate that 
students would like to enroll in more dual credit courses and that dual credit courses were more 
challenging than regular high school classes. Students expressed lower levels of agreement with 
statements that it was easy to communicate with instructors, other course students, and that it was easy to 
get help with difficult content. Students also expressed agreement that dual credit courses have different 
calendars than their high schools, which confirms principals’ and facilitators’ concerns about scheduling 
conflicts that affect when students take exams, have holidays, or receive grades. 

SUMMARY 

About 29% of R-Tech districts offered dual credit programs as part of R-Tech during the program’s first 
year, and about 7% of the 1 1th- and 12th-grade students participating in R-Tech enrolled in dual credit 
courses. Relative to other 1 1th- and 12th-grade students receiving R-Tech services, dual credit students 
were more likely to be White or female, and less likely to be African American or from economically 
disadvantaged backgrounds. Students enrolled in dual credit courses tended to focus on subject areas 
related to social studies (e.g., history, government, economics) and English, and proportionately few 
students took math and science courses for dual credit. 

Principals and facilitators on campuses implementing dual credit programs reported challenges in 
working out differences between district and university calendars and in identifying funds to pay college 
costs for tuition and textbooks. They overcame challenges by collaborating with staff at partner 
universities and adjusting district timelines for dual credit students. Districts also used HSA funding to 
cover the costs of college courses. 

Surveyed students who participated in dual credit courses felt that such courses were more rigorous than 
their regular high school classes and provided strong preparation for postsecondary educational 
opportunities. Students also indicated that they would like to participate in more dual credit courses, 
which suggests districts may want to expand dual credit offerings. 
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Appendix F 

Technical Appendix— Hierarchical Linear Modeling (HLM) 


EFFECTS OF R-TECH ON STUDENTS’ TAKS SCORES (CHAPTER 6) 

The evaluation investigates three aspects of the R-Tech program: (1) the effect of reported R-Tech 
instructional hours per week on students’ TAKS scores, (2) the effect of supplemental vs. non- 
supplemental provision of services, and (3) the effect of program type on students’ TAKS scores. The two 
primary types of R-Tech program implemented during the project’s first year were self-paced computer 
software and dual credit or distance learning. Three other program types were not included in these 
analyses because they were only implemented in two districts apiece. Inclusion would have resulted in 
collinearity issues and could have resulted in the identification of the districts and campuses. 

Analyses 

The effects of R-Tech instructional time, program type, and supplemental status on students’ 
reading/ELA, mathematics, science, and social studies TAKS T scores were analyzed using a two-level 
hierarchical linear model (HLM). HLM is a “value added” methodology. That is, after controlling for 
students’ initial achievement and characteristics and accounting for variance at both the student and 
school level, researchers can assess the “value added” by an indicator like program type or supplemental 
status. Analyses were conducted for students participating in R-Tech in fall 2008 and/or spring 2009. 
Separate analyses were performed for the supplemental status indicator and for program type. 

Student-level model. In the student- level model, spring 2009 TAKS T scores 1 were regressed on spring 
2008 TAKS T scores, average number of R-Tech primary instructional hours per week, 2 economic status 
(0 if not disadvantaged, 1 if disadvantaged), African American status (0 if not African American, 1 if 
African American), Hispanic status (0 if not Hispanic, 1 if Hispanic), gender (0 if male, 1 if female), 
middle school grades (1 if in Grades 6, 7, or 8; 0 if in Grades 9, 10, or 1 1), and LEP status (1 if LEP, 0 if 
not). That is, 

Yy = p 0 j + P ij( Spring 2008 T score [grand mean centered])^ + [h/ R-Tech instructional hours 
per week [grand mean centered ])y + [Up Economic status )y + fi 4 / African American 
status)ij + [h,( Hispanic status )y + ft 6 / Female )y + f 7/ Middle school grades)ij + [ff LEP 
status)^ + r ( y. 


'The specific TAKS test used in the analysis corresponded to a primary, secondary, or tertiary instructional focus 
reported for a student in the fall 2008 or spring 2009 student upload. 

2 The average primary instructional hours per week was calculated by summing the fall 2008 primary instructional 
hours per week and the spring 2009 primary instructional hours per week and dividing by 2, given that fall and 
spring hours were in the 0 to 20 range. The mean of the average primary instructional hours per week was 2.3 with a 
range from 0.01 to 20. The distribution was positively skewed and not normal (Kolmogorov-Smirnov Z = 19.6). For 
example, 43% of participants averaged 1 hour or less per week, and 61% averaged 2 hours or less per week. 
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With 2009 TAKS reading/ELA, mathematics, science, and social studies T scores for R-Tech campuses, 
significant variation was found across schools. Specifically, 14% of the variance in TAKS reading/ELA T 
scores, 17% of the variance in TAKS mathematics T scores, 17% of the variance in TAKS science T 
scores, and 27% of the variance in TAKS social studies T scores was between campuses. 3 Thus, the 
school means (p 0 j) were specified as randomly varying. The coefficients for spring 2008 T scores (pp) and 
R-Tech instructional hours per week (p 2 j) were also specified as randomly varying when chi-square 
statistics were significant. The coefficients for the remaining independent variables were specified as 
fixed. 

School-level model. At the school level, one model was used to answer the question of whether services 
provided outside regular instructional hours or services provided during regular instructional hours 
differentially effected students’ TAKS scores, after controlling for school achievement 4 or the percentage 
of students passing all 2009 TAKS tests (with percentages ranging from 0.0% to 89.0%, and with a grand 
mean of 64.8%), as well as initial achievement, number of R-Tech instructional hours per week, ethnicity, 
economic status, LEP status, gender, and grade level. A second model was developed to answer the 
question of whether program type (self-paced computer software and dual credit or distance learning) had 
an effect on students’ TAKS scores, again net of the control variables. That is, 

fioj = yoo + yoi( Supplemental status [or Program type])j + yo 2 (School achievement [grand mean 

centered] )j + poj- 


Data were analyzed using a two-level HLM. A student was included in the analysis for a particular TAKS 
test if his or her fall 2008 or spring 2009 student upload indicated that the content area was a primary, 
secondary, or tertiary instructional focus. 5 Researchers posit that R-Tech instructional hours, 
supplemental status, and program type, along with school achievement, gender, economically 
disadvantaged status, ethnicity, LEP status, grade grouping, and prior year TAKS score are related to the 
current year TAKS score. Statistical details for the TAKS reading/ELA analyses are provided in Tables 
F.l, F.2, and F.3, for the TAKS mathematics analyses in Tables F.4, F.5, and F.6, for the TAKS science 
analyses in Tables F.7, F.8, and F.9, and for the TAKS social studies analyses in Tables F.10, F.ll, and 
F. 12. 


3 Variation in TAKS scores can be divided between variation over students and variation over schools. The 
percentage of this total variation in TAKS scores that is over schools is reported here. The presence of significant 
variation over schools indicates the need to employ multi-level modeling rather than conventional regression. 

4 Note that when both school poverty or the percentage of economically disadvantaged students at a school (a 
continuous variable with percentages ranging from 4.5% to 100%, and with a grand mean of 52.1%) and 2009 
school achievement were included in the school-level model, coefficient sizes were inflated indicating collinearity 
between the two indicators. School achievement was used as the sole contextual variable because it resulted in the 
greatest reduction in campus level variance. The 2009 school achievement data were preliminary results provided by 
TEA. 

5 In the data uploads hosted by TEA, R-Tech districts submitted information on students' program participation. 
Three data fields in the uploads were the primary, secondary, and tertiary academic or instructional areas of a 
students’ R-Tech program. 
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Table F.l 

Descriptive Statistics for TAKS Reading/ELA Achievement 


Variable Name 

N 

Mean 

SD 

Student-Level Descriptive Statistics: (Level 1) 

Female 

5,524 

0.46 

0.50 

African American 

5,524 

0.07 

0.26 

Flispanic 

5,524 

0.30 

0.46 

Economically disadvantaged (1 = yes, 0 = no) 

5,524 

0.55 

0.50 

Limited English proficient (1 = yes, 0 = no) 

5,524 

0.04 

0.20 

Middle grades (6 to 8 = 1 , 9 to 1 1 = 0) 

5,898 

0.55 

0.50 

R-Tech instructional hours 

5,678 

2.19 

2.14 

TAKS reading/ELA T Score (2008) 

3,816 

48.52 

9.30 

TAKS reading/ELA T Score (2009) 

5,268 

47.79 

9.22 

School-Level Descriptive Statistics: (Level 2) 

School achievement (percentage) 

80 

65.32 

12.96 

Supplemental status (1 = yes, 0 = no) 

80 

0.56 

0.50 

Self-paced software (1 = yes, 0 = no) 

80 

0.90 

0.30 

Distance learning (1 = yes, 0 = no) 

80 

0.25 

0.44 


Sources: Texas Education Agency (TEA) R-Tech Student Upload data: fall 2008 and spring 2009. 
Public Education Information Management System (PEIMS) student demographic data from fall 
2008. Texas Assessment of Knowledge and Skills 2008 and 2009 individual student data from 
TEA. 
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Table F.2 

Hierarchical Regression Models Predicting the Effects of Instructional Time, Program Type, 
and Supplemental Status on TAKS Reading/ELA Achievement 


Group 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t-value 

Supplemental Status 


Intercept 

48.857 

0.529 

92.30*** 


Supplemental status 

-0.795 

0.411 

-1.93 


School achievement 3 

0.011 

0.021 

0.53 


Female 

0.747 

0.233 

3.21** 


African American 

-1.244 

0.547 

-2.27* 


Hispanic 

-0.366 

0.387 

-0.95 


Economic disadvantage 

-0.873 

0.292 

-2.99** 


Limited English proficient 

-3.608 

0.917 

-3 94*** 


Middle school level 0 

0.355 

0.426 

0.83 


R-Tech instructional hours ad 

-0.095 

0.084 

-1.13 


Spring 2008 T score 1 * 

0.498 

0.032 

15.45*** 

Program Type 


Intercept 

49.826 

0.664 

75.03*** 


Self-paced software* 5 

-1.356 

0.591 

-2.29* 


Distance learning* 5 

-0.447 

0.497 

-0.90 


School achievement 3 

0.006 

0.020 

0.32 


Female 

0.756 

0.232 

3.27** 


African American 

-1.375 

0.534 

-2.57* 


Hispanic 

-0.468 

0.389 

-1.21 


Economic disadvantage 

-0.870 

0.293 

-2.97** 


Limited English proficient 

-3.757 

0.906 

-4 15*** 


Middle school level 0 

0.318 

0.428 

0.74 


R-Tech instructional hours 3 '* 1 

-0.150 

0.079 

-1.90 


Spring 2008 T score 3 

0.499 

0.032 

15.40*** 


*p < .05; **p < .01; ***p < .001. 

Sources: Texas Education Agency (TEA) R-Tech Student Upload data: fall 2008 and spring 2009. Public 
Education Information Management System (PEIMS) student demographic data from fall 2008. Texas 
Assessment of Knowledge and Skills 2008 and 2009 individual student data from TEA. 

Notes. Data from the fall 2008 or spring 2009 student upload indicated that English language arts was a 
primary, secondary, or tertiary instructional focus. Analyses included 3,41 1 R-Tech participants from 75 
campuses. Fourteen percent of the variance in Texas Assessment of Knowledge and Skills (TAKS ) 
reading/English language arts scores was between campuses. The percentage of within-school variance 
explained by the student-level predictors was 38%. The percentage of between-school variance explained by 
the campus-level predictors (relative to the student-level model) was 1% for the supplemental status model 
and 9% for the program type model. Note that when both campus poverty (a continuous variable with 
percentages ranging from 4.5% to 100%, and with a grand mean of 52.1%) and campus achievement were 
included in the school-level model, coefficient sizes were inflated indicating collinearity between the two 
indicators. Campus achievement was used as the sole contextual variable because it resulted in the greatest 
reduction in level 2 variance. 

“The predictor was centered around its grand mean. 

b The program types were limited to self-paced computer software and dual credit or distance learning because 
each of the remaining program types (tutoring and homework support online, school-wide technology 
immersion, and use of iPods to deliver content ) was implemented in only two districts. Technically, this 
avoided multicolinearity issues, and, practically, it avoided identifying districts and campuses. 

“The student was in Grades 6, 7, or 8 in 2008-09. 

d Average number of reported R-Tech instructional hours per week. 
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Table F.3 

Variance Decomposition from Conditional HLM Models Showing the Effects of Instructional 
Time, Program Type, and Supplemental Status on TAKS Reading/ELA Achievement 


Test/ 

Variance 




Random Effect 

Component 

df 

X 2 

P 

Supplemental Status 

Level- 1 student effect 

46.8960 


School mean 

2.0169 

61 

175.99 

0.000 

Instructional hours-outcome slope 

Effect not random 

2008 TAKS-outcome slope 

0.0360 

63 

220.02 

0.000 

Program Type 

Level- 1 student effect 

46.9020 


School mean 

1.8516 

60 

184.26 

0.000 

Instructional hours-outcome slope 

Effect not random 

2008 TAKS-outcome slope 

0.0354 

63 

219.27 

0.000 


Sources: Texas Education Agency (TEA) R-Tech Student Upload data: fall 2008 and spring 2009. Public 
Education Information Management System (PEIMS) student demographic data from fall 2008. Texas 
Assessment of Knowledge and Skills 2008 and 2009 individual student data from TEA. 


Table F.4 

Descriptive Statistics for TAKS Mathematics Achievement 


Variable Name 

N 

Mean 

SD 

Student-Level Descriptive Statistics: (Level 1) 

Female 

8,428 

0.48 

0.50 

African American 

8,428 

0.11 

0.31 

Hispanic 

8,428 

0.30 

0.46 

Economically disadvantaged ( 1 = yes, 0 = no) 

8,428 

0.56 

0.50 

Limited English proficient (1 = yes, 0 = no) 

8,428 

0.04 

0.19 

Middle grades (6 to 8 = 1 , 9 to 1 1 = 0) 

8,893 

0.53 

0.50 

R-Tech instructional hours 

8,672 

2.20 

2.42 

TAKS mathematics T score (2008) 

5,867 

47.84 

8.47 

TAKS mathematics T score (2009) 

7,925 

47.76 

8.77 

School-Level Descriptive Statistics: (Level 2) 

School achievement (percentage) 

84 

64.82 

13.31 

Supplemental status ( 1 = yes, 0 = no) 

84 

0.56 

0.50 

Self-paced software (1 = yes, 0 = no) 

84 

0.92 

0.28 

Distance learning (1 = yes, 0 = no) 

84 

0.26 

0.44 


Sources: Texas Education Agency (TEA) R-Tech Student Upload data: fall 2008 and spring 2009. 
Public Education Information Management System (PEIMS) student demographic data from fall 
2008. Texas Assessment of Knowledge and Skills 2008 and 2009 individual student data from TEA. 
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Table F.5 

Hierarchical Regression Models Predicting the Effects of Instructional Time, Program Type, 
and Supplemental Status on TAKS Mathematics Achievement 


Group 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t-value 

Supplemental Status 


Intercept 

48.260 

0.313 

154.10*** 


Supplemental status 

-0.339 

0.340 

-1.00 


School achievement 3 

0.067 

0.015 

4 38*** 


Female 

-0.060 

0.150 

-0.40 


African American 

-0.326 

0.295 

-1.11 


Hispanic 

0.047 

0.245 

0.19 


Economic disadvantage 

-0.490 

0.186 

-2.64** 


Limited English proficient 

-1.408 

0.556 

-2.53** 


Middle school level 0 

0.083 

0.325 

0.26 


R-Tech instructional hours 3 * 3 

-0.054 

0.083 

-0.65 


Spring 2008 T score 1 * 

0.699 

0.026 

26.65*** 

Program Type 


Intercept 

47.620 

0.522 

91.16*** 


Self-paced software 15 

0.457 

0.481 

0.95 


Distance learning 15 

0.286 

0.330 

0.87 


School achievement 3 

0.072 

0.015 

4 80*** 


Female 

-0.059 

0.151 

-0.39 


African American 

-0.346 

0.298 

-1.16 


Hispanic 

0.018 

0.241 

0.07 


Economic Disadvantage 

-0.487 

0.185 

-2.63** 


Limited English proficient 

-1.423 

0.565 

-2.52* 


Middle school level 0 

0.000 

0.332 

0.00 


R-Tech instructional hours 3 '* 3 

-0.044 

0.095 

-0.46 


Spring 2008 T score 3 

0.698 

0.026 

26.42*** 


*p < .05; **p < .01; ***p < .001. 

Sources: Texas Education Agency (TEA) R-Tech Student Upload data: fall 2008 and spring 2009. Public 
Education Information Management System (PEIMS) student demographic data from fall 2008. Texas 
Assessment of Knowledge and Skills 2008 and 2009 individual student data from TEA. 

Notes. Data from the fall 2008 or spring 2009 student upload indicated that mathematics was a primary, 
secondary, or tertiary instructional focus. Analyses included 5,334 R-Tech participants from 78 campuses. 
Seventeen percent of the variance in Texas Assessment of Knowledge and Skills (TAKS) mathematics scores 
was between campuses. The percentage of within-school variance explained by the student-level predictors 
was 59%. The percentage of between-school variance explained by the campus-level predictors (relative to 
the student-level model) was 45% for the supplemental status model and 39% for the program type model. 
Note that when both campus poverty (a continuous variable with percentages ranging from 4.5% to 100%, 
and with a grand mean of 52.1%) and campus achievement were included in the school-level model, 
coefficient sizes were inflated indicating collinearity between the two indicators. Campus achievement was 
used as the sole contextual variable because it resulted in the greatest reduction in level 2 variance. 

“The predictor was centered around its grand mean. 

b The program types were limited to self-paced computer software and dual credit or distance learning because 
each of the remaining program types (tutoring and homework support online, school-wide technology 
immersion, and use of iPods to deliver content ) was implemented in only two districts. Technically, this 
avoided multicolinearity issues, and, practically, it avoided identifying districts and campuses. 

“The student was in Grades 6, 7, or 8 in 2008-09. 

“'Average number of reported R-Tech instructional hours per week. 
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Table F.6 

Variance Decomposition from Conditional HLM Models Showing the Effects of Instructional 
Time, Program Type, and Supplemental Status on TAKS Mathematics Achievement 


Test/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Supplemental Status 

Level- 1 student effect 

27.0093 


School mean 

1.2474 

61 

154.54 

0.000 

Instructional hours-outcome slope 

0.1693 

63 

118.00 

0.000 

2008 TAKS-outcome slope 

0.0329 

63 

399.44 

0.000 

Program Type 

Level- 1 student effect 

26.9878 


School mean 

1.3867 

60 

159.60 

0.000 

Instructional hours-outcome slope 

0.2198 

63 

117.87 

0.000 

2008 TAKS -outcome slope 

0.0333 

63 

400.72 

0.000 


Sources: Texas Education Agency (TEA) R-Tech Student Upload data: fall 2008 and spring 2009. Public 
Education Information Management System (PEIMS) student demographic data from fall 2008. Texas 
Assessment of Knowledge and Skills 2008 and 2009 individual student data from TEA. 


Table F.7 

Descriptive Statistics for TAKS Science Achievement 


Variable Name 

N 

Mean 

SD 

Student-Level Descriptive Statistics: (Level 1) 

Lemale 

865 

0.47 

0.50 

African American 

865 

0.09 

0.28 

Hispanic 

865 

0.28 

0.45 

Economic disadvantage (1 = yes, 0 = no) 

865 

0.45 

0.50 

Limited English proficient (1 = yes, 0 = no) 

865 

0.03 

0.17 

R-Tech instructional hours 

892 

2.60 

2.37 

TAKS science T score (2008) 

686 

48.84 

8.94 

TAKS science T score (2009) 

732 

48.48 

9.49 

School-Level Descriptive Statistics: (Level 2) 

School achievement (percentage) 

33 

62.60 

15.88 

Supplemental status (1 = yes, 0 = no) 

33 

0.52 

0.51 

Self-paced software (1 = yes, 0 = no) 

33 

0.91 

0.29 

Distance learning (1 = yes, 0 = no) 

33 

0.27 

0.45 


Sources: Texas Education Agency (TEA) R-Tech Student Upload data: fall 2008 and spring 2009. 
Public Education Information Management System (PEIMS) student demographic data from fall 
2008. Texas Assessment of Knowledge and Skills 2008 and 2009 individual student data from 
TEA. 
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Table F.8 

Hierarchical Regression Models Predicting the Effects of Instructional Time, Program Type, 
and Supplemental Status on TAKS Science Achievement 3 


School-Level 

Gamma 

Standard 


Group 

Analysis 

Coefficient 

Error 

t- value 

Supplemental Status | 


Intercept 

49.792 

1.043 

47 72*** 


Supplemental status 

-1.692 

1.039 

-1.63 


School achievement 3 

0.007 

0.071 

0.10 


Female 

0.350 

0.498 

0.70 


African American 

0.847 

0.757 

1.12 


Hispanic 

-0.804 

0.422 

-1.91 


Economic disadvantage 

-0.171 

0.450 

-0.38 


Limited English proficient 

-0.872 

1.265 

-0.69 


R-Tech instructional hours 3 ' 0 

0.324 

0.275 

1.18 


Spring 2008 T score 3 

0.799 

0.046 

17.51*** 

Program Type | 


Intercept 

48.197 

1.282 

37.59*** 


Self-paced software 13 

0.772 

1.254 

0.62 


School achievement 3 

0.035 

0.070 

0.50 


Female 

0.373 

0.500 

0.75 


African American 

0.786 

0.756 

1.04 


Hispanic 

-0.887 

0.409 

-2.17* 


Economic disadvantage 

-0.165 

0.449 

-0.37 


Limited English proficient 

-0.841 

1.256 

-0.67 


R-Tech instructional hours 3 ' 0 

0.411 

0.271 

1.52 


Spring 2008 T score 3 

0.801 

0.046 

17 39 *** 


*p < .05; **p < .01; ***p < .001. 

Sources: Texas Education Agency (TEA) R-Tech Student Upload data; fall 2008 and spring 2009. Public 
Education Information Management System (PEIMS) student demographic data from fall 2008. Texas 
Assessment of Knowledge and Skills 2008 and 2009 individual student data from TEA. 

Notes. Data from the fall 2008 or spring 2009 student upload indicated that science was a primary, secondary, 
or tertiary instructional focus. Analyses included 609 R-Tech participants from 29 campuses. Seventeen 
percent of the variance in Texas Assessment of Knowledge and Skills (TAKS ) science scores was between 
campuses. The percentage of within-school variance explained by the student-level predictors was 70%. The 
percentage of between-school variance explained by the campus-level predictors (relative to the student-level 
model) was 15% for the supplemental status model and 0% for the program type model. Note that when both 
campus poverty (a continuous variable with percentages ranging from 4.5% to 100%, and with a grand mean of 
52.1%) and campus achievement were included in the school-level model, coefficient sizes were inflated 
indicating collinearity between the two indicators. Campus achievement was used as the sole contextual 
variable because it resulted in the greatest reduction in level 2 variance. 

“The predictor was centered around its grand mean. 

b Program type was limited to self-paced computer software because only a small number of the Grade 1 1 
science students were exclusively in a dual credit or distance learning program. 

“Average number of reported R-Tech instructional hours per week. 
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Table F.9 

Variance Decomposition from Conditional HLM Models Showing the Effects of Instructional 
Time, Program Type, and Supplemental Status on TAKS Science Achievement 


Test/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Supplemental Status 

Level- 1 student effect 

22.6931 




School mean 

4.4767 

13 

30.36 

0.004 

Instructional hours-outcome slope 

0.8218 

15 

37.74 

0.001 

2008 TAKS -outcome slope 

0.0318 

15 

43.32 

0.000 

Program Type 

Level- 1 student effect 

22.6952 




School mean 

5.5274 

13 

33.24 

0.002 

Instructional hours-outcome slope 

0.8219 

15 

41.08 

0.000 

2008 TAKS -outcome slope 

0.0330 

15 

43.32 

0.000 


Sources: Texas Education Agency (TEA) R-Tech Student Upload data: fall 2008 and spring 2009. Public 
Education Information Management System (PEIMS) student demographic data from fall 2008. Texas 
Assessment of Knowledge and Skills 2008 and 2009 individual student data from TEA. 


Table F.10 

Descriptive Statistics for TAKS Social Studies Achievement 


Variable Name 

N 

Mean 

SD 

Student-Level Descriptive Statistics: (Level 1) 

Lemale 

418 

0.49 

0.50 

African American 

418 

0.06 

0.24 

Hispanic 

418 

0.25 

0.43 

Economic disadvantage (1 = yes, 0 = no) 

418 

0.30 

0.46 

Limited English proficient (1 = yes, 0 = no) 

418 

0.01 

0.11 

R-Tech instructional hours 

432 

3.07 

2.41 

TAKS social studies T score (2008) 

361 

51.41 

9.81 

TAKS social studies T score (2009) 

342 

51.65 

10.12 

School-Level Descriptive Statistics: (Level 2) 

School achievement (percentage) 

33 

62.88 

15.31 

Supplemental status (1 = yes, 0 = no) 

33 

0.55 

0.51 

Self-paced software (1 = yes, 0 = no) 

33 

0.91 

0.29 

Distance learning (1 = yes, 0 = no) 

33 

0.33 

0.48 


Sources: Texas Education Agency (TEA) R-Tech Student Upload data: fall 2008 and spring 2009. 
Public Education Information Management System (PEIMS) student demographic data from fall 
2008. Texas Assessment of Knowledge and Skills 2008 and 2009 individual student data from TEA. 
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Table F.ll 

Hierarchical Regression Models Predicting the Effects of Instructional Time, Program Type, 
and Supplemental Status on TAKS Social Studies Achievement 3 


School-Level 

Gamma 

Standard 


Group 

Analysis 

Coefficient 

Error 

t-value 

Supplemental Status 


Intercept 

53.513 

0.875 

61.17*** 


Supplemental status 

-1.674 

0.620 

-2.70* 


School Achievement 2 

0.007 

0.041 

0.18 


Female 

-1.682 

0.667 

-2.52* 


African American 

-1.985 

1.227 

-1.62 


Hispanic 

-1.614 

0.940 

-1.72 


Economic disadvantage 

-0.267 

0.877 

-0.31 


Limited English proficient 

-3.645 

3.668 

-0.99 


R-Tech instructional hours 2 ' 0 

-0.372 

0.202 

-1.84 


Spring 2008 T score 2 

0.758 

0.048 

15.86*** 

Program Type 


Intercept 

52.183 

0.950 

54.92*** 


Self-paced software 13 

0.421 

0.855 

0.49 


School achievement 2 

0.045 

0.045 

0.99 


Female 

-1.662 

0.671 

-2.48* 


African American 

-1.587 

1.277 

-1.24 


Hispanic 

-1.564 

0.972 

-1.61 


Economic disadvantage 

-0.361 

0.890 

-0.41 


Limited English proficient 

-3.481 

3.799 

-0.92 


R-Tech instructional hours 2 ' 0 

-0.345 

0.185 

-1.87 


Spring 2008 T score 2 

0.749 

0.046 

16.34*** 


*p < .05; **p < .01; ***p < .001. 

Sources: Texas Education Agency (TEA) R-Tech Student Upload data; fall 2008 and spring 2009. Public 
Education Information Management System (PEIMS) student demographic data from fall 2008. Texas 
Assessment of Knowledge and Skills 2008 and 2009 individual student data from TEA. 

Notes. Data from the fall 2008 or spring 2009 student upload indicated that social studies was a primary, 
secondary, or tertiary instructional focus. Analyses included 307 R-Tech participants from 26 campuses. 
Twenty-seven percent of the variance in Texas Assessment of Knowledge and Skills (TAKS) social studies 
scores was between campuses. The percentage of within-school variance explained by the student-level 
predictors was 50%. The percentage of between-school variance explained by the campus-level predictors 
(relative to the student-level model) was 0% for both the supplemental status and program type models. 
“The predictor was centered around its grand mean. 

b Program type was limited to self-paced computer software because only a small number of the Grade 1 1 
social studies students were exclusively in a dual credit or distance learning program. 

“Average number of reported R-Tech instructional hours per week. 
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Table F.12 

Variance Decomposition from Conditional HLM Models Showing the Effects of Instructional 
Time, Program Type, and Supplemental Status on TAKS Social Studies Achievement 


Test/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Supplemental Status 

Fevel- 1 student effect 

36.3546 




School mean 

1.9311 

16 

22.91 

0.116 

Instructional hours-outcome Slope 

Effect Not Random 

2008 TAKS -outcome Slope 

0.0175 

18 

26.97 

0.079 

Program Type 

Fevel- 1 student effect 

36.7378 




School mean 

2.0722 

16 

23.34 

0.105 

Instructional hours-outcome slope 

Effect Not Random 

2008 TAKS -outcome slope 

0.0114 

18 

26.58 

0.087 

Sources: Texas Education Agency (TEA) R-Tech Student Upload data: fall 2008 and spring 2009 

Public 


Education Information Management System (PEIMS) student demographic data from fall 2008. Texas 
Assessment of Knowledge and Skills 2008 and 2009 individual student data from TEA. 

HLM Analyses Limitations 

The goal of analyses is not to generalize the results to all students who participated in R-Tech programs in 
all school districts. Rather, the purpose is to generalize the results to the target population of students who 
took part in R-Tech instructional activities in the 115 participating campuses from the 62 participating 
school districts. Yet, even with this rather modest aim, there are several potential limitations to the 
generalizability of the findings. These are discussed in the sections that follow. 

Missing data. Consider that there were 14,217 students who participated in R-Tech activities in fall 2008 
or spring 2009. Of these, 12,786 were in Grades 6 through 1 1 in 2008-09. Of the Grades 6 through 1 1 
students, 5,936 reported reading/ELA as an instructional focus, and 9,165 reported mathematics as an 
instructional focus (see Table F.13). In addition, 900 Grade 1 1 students reported science as an 
instructional focus, and 444 reported social studies as an instructional focus (Table F.13). Table F.13 
reports the numbers of these students having valid data at each step in the construction of the data files 
used in the HFM analyses. It is apparent that survivorship complicates student-level analyses. Student 
cohort membership declined at each step in the construction of data files. For example, of the 5,936 
students who reported reading/EFA as an instructional focus, 5,559 had valid demographic data (a 
reduction of 377 or 6.4%), 5,545 reported valid fall 2008 or spring 2009 instructional hours (a reduction 
of 14 or 0.3%), 3,624 had valid 2008 and 2009 reading/EFA TAKS scores (a reduction of 1,921 or 
34.6%), and 3,41 1 had valid campus supplemental status and program type data (a reduction of 213 or 
5.9%). Overall, the original 5,936 sample of students with reading/EFA as an instructional focus was 
reduced by 2,525 or 42.5%. A major portion of this reduction (32.4%) was due to students not having 
valid TAKS scores in both spring 2008 and spring 2009. Similarly, the original sample of 9,165 students 
who reported mathematics as an instructional focus was reduced by 3,831 or 41.8%. The reduction in the 
Grade 1 1 science sample was 291 students or 32.3%, and the reduction in the Grade 1 1 social studies 
sample was 137 students or 30.9%. 
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Table F.13 

Number of Cases at Each Step in the HLM Analyses 


Steps 

Reading/ 

ELA 

Mathematics 

Science 

Social 

Studies 

Subject as a primary, secondary, or 
tertiary instructional focus for R-Tech 
activities in fall 2008 or spring 2009 

5,936 

9,165 

900 

444 

Valid gender, ethnic, economic, and 
LEP data 

5,559 

8,658 

869 

429 

Valid R-Tech instructional hours in 
fall 2008 or spring 2009 

5,545 

8,529 

864 

428 

Valid TAKS scores in spring 2008 
and spring 2009 

3,624 

5,576 

611 

316 

Valid campus supplemental status 
and program type data 

3,411 

5,334 

609 

307 

Percentage reduction 

42.5% 

41.8% 

32.3% 

30.9% 


When missing data are an issue, researchers must ask whether the surviving samples used in the HLM 
analyses are representative of the original samples. Do R-Tech students who have complete data (i.e., 
possibly indicating continuous enrollment) resemble the entire R-Tech student population? (This is 
especially relevant when considering programs with high turnover rates, such as credit recovery 
programs. Many R-Tech programs fit this category.) Table F.14 compares the original samples with the 
restricted samples used in each of the HLM analyses. Gender differences between the two samples were 
slight as were ethnic differences for the reading/ELA and mathematics analyses. The restricted science 
and social studies samples had lower percentages of minority students. The restricted reading/ELA, 
mathematics, and science samples had lower percentages of LEP students. All of the restricted samples 
had lower percentages of economically disadvantaged students and students participating in special 
education services. Thus, the restricted samples were different than the original samples — they were less 
likely to be economically disadvantaged, LEP, and special education students. 


Table F.14 

Demographic Characteristics of Full and Restricted or Partial Samples 



Reading/ELA 

Mathematics 

Science 

Social Studies 

Characteristic 

Full 

Partial 

Full 

Partial 

Full 

Partial 

Full 

Partial 

Percentage minority 

37.3% 

37.1% 

41.2% 

41.0% 

36.9% 

32.0% 

31.9% 

23.8% 

Percentage female 

46.1% 

47.1% 

48.4% 

49.2% 

47.0% 

48.4% 

50.1% 

49.8% 

Percentage disadvantaged 

55.0% 

52.5% 

55.9% 

53.5% 

45.2% 

38.4% 

30.8% 

23.8% 

Percentage Limited English 
proficient 

4.2% 

2.9% 

3.8% 

2.5% 

3.0% 

1.6% 

1.2% 

1.3% 

Percentage special education 

16.4% 

3.8% 

13.0% 

2.6% 

15.1% 

3.0% 

8.9% 

2.6% 


TAKS as the measure of student achievement. Because the TAKS is not a vertically equated test (i.e., 
the skills measured and the scoring from one grade to the next is along a continuum), results arc not 
comparable from grade to grade and from year to year. Thus, researchers used standard scores (T scores) 
to compare students from one year to the next. These scores allow for normative comparisons (where 
students fall in the distribution of test scores from one year to the next), but not for criterion-referenced 
comparisons (where students fall on a proficiency scale from one year to the next). 
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TAKS does not measure science and social studies at each of the tested grade levels. Thus, to include a 
prior year TAKS test, science and social studies samples included only Grade 1 1 students. This resulted 
in relatively small numbers of students in the science and social studies analyses. 
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